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Abstract. An orbitope is the convex hull of an orbit of a compact group acting linearly 
on a vector space. These highly symmetric convex bodies lie at the crossroads of several 
fields including convex geometry, algebraic geometry, and optimization. We present a self- 
contained theory of orbitopes with particular emphasis on instances arising from the groups 
SO(n) and 0(n). These include Schur-Horn orbitopes, tautological orbitopes, Caratheodory 
orbitopes, Veronese orbitopes, and Grassmann orbitopes. We study their face lattices, their 
algebraic boundaries, and representations as spectrahedra or projected spectrahedra. 



1. Introduction 

An orbitope is the convex hull of an orbit of a compact algebraic group G acting linearly on 
a real vector space. The orbit has the structure of a real algebraic variety, and the orbitope 
is a convex, semi-algebraic set. Thus, the study of algebraic orbitopes lies at the heart of 
convex algebraic geometry - the fusion of convex geometry and (real) algebraic geometry. 

Orbitopes have appeared in many contexts in mathematics and its applications. Orbitopes 
of finite groups are highly symmetric convex polytopes which include the platonic solids, 
permutahedra, Birkhoff polytopes, and other favorites from Ziegler's text book [38J, as well 
as the Coxeter orbihedra studied by McCarthy, Ogilvie, Zobin, and Zobin [25]. Farran 
and Robertson's regular convex bodies [IT] are orbitopal generalzations of regular polytopes, 
which were classified by Madden and Robertson [23]. Orbitopes for compact Lie groups, such 
as SO(n), have appeared in investigations ranging from protein structure prediction [23J and 
quantum information [2] to calibrated geometries [16]. Barvinok and Blekherman studied 
the volumes of convex bodies dual to certain S'0(n)-orbitopes, and they concluded that there 
are many more non- negative polynomials than sums of squares [3]- 

This paper initiates the study of orbitopes as geometric objects in their own right. The 
questions we ask about orbitopes originate from three different perspectives: convexity, alge- 
braic geometry, and optimization. In convexity, one would seek to characterize all faces of an 
orbitope. In algebraic geometry, one would examine the Zariski closure of its boundary and 
identify the components and singularities of that hypersurface. In optimization, one would 
ask whether the orbitope is a spectrahedron or the projection of a spectrahedron. 

Spectrahedra are to semidefinite programming what polyhedra are to linear programming. 
More precisely, a spectrahedron is the intersection of the cone of positive semidefinite matrices 



Research of Sanyal supported by a Miller Research Fellowship at UC Berkeley. 
Research of Sottile supported in part by NSF grants DMS-070105 and DMS-1001615. 
Research of Sturmfels supported in part by NSF grant DMS-0757207. 



1 



2 



RAMAN SANYAL, FRANK SOTTILE, AND BERND STURMFELS 



with an affine space. It can be represented as the set of points x G M™ such that 
(1.1) A + x x A x + • • • + x n A n y 0, 

where A ,Ai,...,A n are symmetric matrices and y denotes positive semidefiniteness. 
From a spectrahedral description many geometric properties, both convex and algebraic, are 
within reach. Furthermore, if an orbitope admits a representation (11 .11) then it is easy to 
maximize or minimize a linear function over that orbitope. Here is a simple illustration. 

Example 1.1. Consider the action of the group G = SO(2) on the space Sym 4 (IR 2 ) ~ R 5 of 
binary quartics and take the convex hull of the orbit of v = x 4 . The four-dimensional convex 
body conv(G • v) is a Caratheodory orbitope. This orbitope is a spectrahedron: it coincides 
with the set of all binary quartics Ao£ 4 + 4Aix 3 y + 6A2X 2 ?/ 2 + 4A3X?/ 3 + A4?/ 4 such that 

(Ao Ai A 2 \ 
Ai A 2 A 3 ^ and A + 2A 2 + A 4 = 1. 
A2 A3 X^J 

This representation (jl.2p will be derived in Section 5, where we will also see that it is 
equivalent to classical results from the theory of positive polynomials [32] • The Hankel 
matrix shows that the boundary of conv(G ■ v) is an irreducible cubic hypersurface in R 4 , 
defined by the vanishing of the Hankel determinant. It also reveals that this four- dimensional 
Caratheodory orbitope is 2-neighborly: the extreme points are the rank one matrices, and 
any two of them are connected by an edge. The typical intersection of conv(G-t') with a three- 
dimensional affine plane looks like an inflated tetrahedron. This three-dimensional convex 
body is bounded by Cayley's cubic surface, shown in Figured] Alternative pictures of this 
convex body can be found in [27] Fig. 3] and [351 Fig- 4]. The four vertices of the tetrahedron 
lie on the curve G ■ v, and its six edges are inclusion-maximal faces of conv(G • v ). □ 




Figure 1. Cross-section of a four-dimensional Caratheodory orbitope. 

This article is organized as follows. We begin by deriving the basic definitions and a 
few general results about orbitopes, and we formulate ten key questions which will guide 
our subsequent investigations. These are organized along the themes of convex geometry 
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(Subsection 2.1), algebraic geometry (Subsection 2.2) and optimization (Subsection 2.3). 
These questions are difficult. They are meant to motivate the reader and to guide our 
investigations. We are not yet able to offer a complete solution to any of these ten problems. 

Section 3 is concerned with the action of 0(n) and SO(n) by conjugation on nxn-matrices. 
These decompose into the actions by conjugation on symmetric and skew-symmetric matri- 
ces. The resulting Schur-Horn orbitopes are shown be spectrahedra, their algebraic boundary 
is computed, and their face lattices are derived from certain polytopes known as permuta- 
hedra. The spectrahedral representations of the Schur-Horn orbitopes, stated in Theorems 
13.41 and 13.151 rely on certain Schur functors also known as the additive compound matrices. 

Given a compact real algebraic group of nxn-matrices, we can form its convex hull in 
R nxn . The resulting convex bodies are the tautological orbitopes. In Subsection 4.2 we study 
the tautological orbitope and its dual coorbitope for the group 0{n). Both are spectrahedra, 
characterized by constraints on singular values, and they are unit balls for the operator 
and nuclear matrix norm considered in [31]. Subsections 4.1 and 4.4 are devoted to the 
tautological orbitope for SO(n). A characterization of its faces is given in Theorem 14.111 

The orbitopes of the group 5*0(2) are the convex hulls of trigonometric curves, a classical 
topic initiated by Caratheodory in [8J, further developed in [SUM], and our primary focus in 
Section 5. These S'0(2)-orbitopes can be represented as projections of spectrahedra in two 
distinct ways: in terms of Hermitian Toeplitz matrices or in terms of Hankel matrices. 

A natural generalization of the rational normal curves in Section 5 are the Veronese vari- 
eties. Their convex hulls, the Veronese orbitopes, are dual to the cones of polynomials that 
are non-negative on M. n , as seen in [U [7J [32]. In Section 6 we undertake a detailed study of 
the 15-dimensional Veronese orbitope and its coorbitope which arise from ternary quartics. 

In Section 7 we mesh our investigations with a line of research in differential geometry. 
The Grassmann orbitope is the convex hull of the oriented Grassmann variety in its Plucker 
embedding in the unit sphere in A^R™. This vector space is the d-th exterior power of R n 

and it is isomorphic to r(<*). The facial structure of Grassmann orbitopes has been studied 
in the theory of calibrated manifolds [161 El [26] . We take a fresh look at these orbitopes 
from the point of view of convex algebraic geometry. Theorem 17.31 furnishes a spectrahedral 
representation and the algebraic boundary in the special case d = 2, while Theorem 17.61 
shows that the Grassmann orbitopes fail to be spectrahedra in general. 



2. Setup, Tools and Questions 

Let G be a real, compact, linear algebraic group, that is, a compact subgroup of GL(n,M) 
for some n G N given as a subvariety. Prototypic examples are the special orthogonal group 
SO(n) — {g e GL(n, R) : gg T = Id„ and det(g) = 1} and the unitary group U(n) = {g e 
GL(n, C) : gg T = Id n }. A real representation of G is a group homomorphism p : G — > GL(V) 
for some finite dimensional real vector space V. We will write g ■ v := p(g)(v) for g G G 
and v £ V. The representation is rational if p is a rational map of algebraic varieties. By 
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choosing an inner product ( • , • ) on V, we may define a G-invariant inner product as 



Here /x denotes the Haar measure, which is the unique G-invariant probability measure 
on G. Choosing coordinates on V so that (•, -)q becomes the standard inner product on 
V ~ R m , we can identify the matrix group G with a subgroup of the orthogonal group 
0{m) = {geGL{m,R):gg T = ld m }. 

The orbit of a vector v G V under the compact group G is the set G ■ v = {g ■ v : g G G}. 
This is a bounded subset of V. The orbit G ■ v is a smooth, compact real algebraic variety 
of dimension 



Here staba(v) = {g G G : g ■ v = v} is the stabilizer of the vector v. In particular, the orbit 
is isomorphic as a G- variety to the compact homogeneous space G/stabc^)- 

The orbitope of G with respect to the vector v G V is the semialgebraic convex body 



We tacitly assume that the group G and its representation p are clear from the context and 
sometimes we write O v or O for conv(G • v). The dimension of an orbitope conv(G • v) is the 
dimension of the affine hull of the orbit G ■ v. 

For small n, the connected subgroups of the orthogonal group 0(n) are known. This leads 
to the following census of low-dimensional orbitopes of connected groups. 

Example 2.1 (The orbitopes of dimension at most four arising from connected groups). 
We identify G with a subgroup of SO(n). We assume O = conv(G • v) is an n-dimensional 
orbitope in R n . This implies that G fixes no non-zero vector. Here is our census: 

n — 1: There are no one-dimensional orbitopes because SO(l) is a point. 

It is known that every proper connected subgroup of 5*0(2) or 5*0(3) fixes a non-zero vector, 
so for n < 3, the subgroup G must be equal to SO(n). This establishes the next two cases: 

n = 2: The only orbitopes in M 2 are the discs {(x, y) G M 2 : x 2 + y 2 < r 2 }. 

n = 3: The only orbitopes in R 3 are the balls {(x, y, z) G R 3 : x 2 + y 2 + z 2 < r 2 }. 

The four- dimensional case is where things begin to get interesting: 

n = 4: The group 50(4) has connected subgroups G of dimension 1, 2, 3 and 6. 

• If dim(G) = 6 then G = 50(4) and the orbitopes are balls in R 4 . 

• If dim(G) = 3 then G ~ SU(2), acting as the unit quaternions on all quaternions, 
HI = M 4 , by either left or right multiplication. Here the orbitopes are also balls in R 4 . 

• If dim(G) = 2 then G ~ 50(2) x 50(2). These tori act on M 4 through an orthogonal 
direct sum decomposition R 2 © R 2 and their orbitopes are products of two discs. 

• If dim(G) = 1 then G ~ 50(2) and we obtain four-dimensional orbitopes that are 
isomorphic, for some positive integers a < b, to the Caratheodory orbitopes 



(2.1) 




dim G • v = dim G — dim stabg (v) . 



conv(G ■ v) 



conv{g • v : g G G} C V. 



C atb := conv 



{ (cos at, sin at, cos bt, sin bt) G R 4 \ t G [0, 2n] } . 
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These orbitopes were introduced one century ago by Caratheodory |8J. Their study 
was picked up in the 1980s by Smilansky [33] and recently by Barvinok and Novik [5]. 
We note that C\$ is affinely isomorphic to the Hankel orbitope in Example 11.11 □ 

To compute the dimensions of orbitopes in general we shall need a pinch of representation 
theory [13]. A representation V of the group G is irreducible if its only subrepresentations 
are {0} and V. If V and W are irreducible representations, then the space Homc(V / ,W / ) 
of equivariant linear maps between them is zero unless V ~ W. Schur's Lemma states 
that EndcfV) := B.om.G(V,V) is a division algebra over R, that is, either M, C, or H. 
If W±, W2, ... is a complete list of distinct irreducible representations of G, and V is any 
representation of G, then we have a canonical decomposition into isotypical representations: 

(2.2) @ftom G (W i ,V)®v DAem ) w i V - 

This is an isomorphism of G-modules. The map (12. 2p on each summand is ((p,w) 1— >■ <p(w). 
The image of the ith summand in V is called the Wi-isotypical component of V, and when 
it is non-zero, we say that the irreducible representation Wi appears in the G-module V. We 
say that V is multiplicity-free if each irreducible representation Wi appears in V at most 
once, so that HomG(Wj, V) has rank 1 or over Endc (Wi). 

Suppose that V contains the trivial representation and write V = M. 1 © V, where M. 1 is the 
trivial isotypical component of V and V does not contain the trivial representation. Any 
vector v G V can be written as v = v q © v', where Vq G M' and v' G V. Then 

G • v = Vo © G • v' and conv(G • v) = Vq © conv(G • v') . 

Thus we lose no geometric information in assuming that V does not contain the trivial 
representation. This property ensures that the linear span of an orbit coincides with its affine 
span. Hence the affine span of an orbitope decomposes along its isotypical components: 

aS(G • v) = @ aS(G ■ vi) for vectors v = ®iVi G V as in 

i>0 

To determine the dimension of aff(G • v) for v in a single isotypical component V we proceed 
as follows. Let V = W l with W irreducible. Then v = (w\, . . . ,wi) and the affine span of 
G-v is isomorphic to W k , where k is the rank of the Endc(V)-module spanned by wi, . . . , Wj. 
Hence, the dimension of aS(G-v ) over K. equals k-dimW. In particular, k < dimE n d G (v)(^ / ) • 
If V is multiplicity free and v has a nonzero projection into each isotypical component of V, 
then dimconv(G • v) = dimV^. 

We see this in the Caratheodory orbitopes for the group SO (2) of 2 x 2 rotation matrices. 
Its nontrivial representations are W a ~ M 2 , where a rotation matric acts through its ath 
power, for a > 0. Let C a ^ be the orbitope of SO (2) with respect to a general vector 
v G W a © Wb- If a 7^ b, then V has two isotypical components and C a j, has dimension four. 
If a = b, then V ~ W% consists of a single isotypical component and C a ^ a is two-dimensional, 
as Endso(2)(IR 2 ) = C, the span of (wi,wz) is complex one-dimensional. 
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2.1. Convex geometry. Orbitopes are convex bodies, and it is natural to begin their study 
from the perspective of classical convexity. A point p in a convex body K C V is an extreme 
point if conv(K\{p}) ^ K. Thus, the set E of extreme points of K is the minimal subset 
satisfying conv(E') = K. An extrinsic description of K is given by its support function 

h{K, ■ ) : V* ->■ K, £ ^ := max{!(i) : a; G fT}. 

In terms of the support function, the convex body K is the set of points x & V such that 
< h(K; t) for every I G V*. Each linear functional £ G V* defines an exposed face of i^: 

AT< = {pe K :£(p) = h(K;£)}. 

An exposed face X is itself a convex body of dimension dimaff(i^). An exposed face of 
dimension is called an exposed point of K. It follows that every exposed point is extreme, 
but the inclusion is typically strict. However, for orbitopes, these two notions coincide. 

Proposition 2.2. Every point in the orbit G-v is exposed in its convex hull. In particular, 
every extreme point of the orbitope conv(G ■ v) is an exposed point. 

Proof. Since G acts orthogonally on V, the orbit G ■ v lies entirely in the sphere in V that 
is centered at and contains the point v. As every point of the sphere is exposed, the entire 
orbit consists of exposed points and hence extreme points. □ 

A closed subset F C K is a face if F contains the two endpoints to any open segment 
in K it intersects. This includes and K itself. An inclusion-maximal proper face of K is 
called a facet. By separation, every face is contained in an exposed face and thus facets are 
automatically exposed. In general, every exposed face is a face but not conversely. 

Question 1. When does an orbitope have only exposed faces? 

The exposed faces of a convex body form a partially ordered set with respect to inclusion, 
called the face lattice. The face lattice is atomic but in general not coatomic as was pointed 
out to us by Stephan Weis. A necessary condition is that the polar body (see below) has 
all faces exposed (cf. [37]). Also, it is generally not graded because "being a face of" is 
not a transitive relation. For example, the four- dimensional Barvinok-Novik orbitope in 
Section 15.11 has triangular exposed faces for which the three vertices are exposed but the 
edges are not. Similar behavior is seen in the convex body on the right of Figure [31 which 
has two triangular exposed facets whose edges and two of three vertices are not exposed. 

Question 2. Describe the face lattices of orbitopes. 

For an orbitope O = conv(G • v), the faces come in G-orbits and these G-orbits come in 
algebraic families. In particular, the zero-dimensional faces come in a family parametrized 
by G. The description of these families is the point of Question [2j For instance, the orbitope 
in Example 11.11 is a four- dimensional, two-neighborly convex body. Its exposed points are 
parametrized by the circle S 1 and the edges come in a two-dimensional family. 

The polar body 

0° = {£eV* : h(0;l) < 1} 
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is called the coorbitope of G with respect to v G V. Our assumption that V does not contain 
the trivial representation ensures that is the centroid of O, and this implies (O ) = O. 
We shall also make use of the cone over the coorbitope 0°. This is the coorbitope cone 

(2.3) 6° = {(£,m) G V* ©M : h(0;£) < m}. 

For a convex body K the assignment \\x\\k '■= inf{A > : Xx G K} defines an (asymmet- 
ric) norm on V with unit ball K. If if is centrally-symmetric with respect to the origin, 
then || ■ || x is an actual norm. In that case the polar body K° is also centrally-symmetric 
and || • \\k° is the dual norm. Norms and support functions are related by 

\\£\\ K o = h(K;£) for all £ G V*. 

In particular, if K is an orbitope, then || • ||# and || • \\k° are G-equivariant norms. 

Every point p in a convex body K is in the convex hull of finitely many extreme points. 
We denote by d p the least cardinality of a set E of extreme points with p G conv(S). We 
call c(if) := sup{<ip : p G K} the Caratheodory number of K. Caratheodory's Theorem 
(see e.g. [3] §1.2]) asserts that c(K) is bounded from above by dim if + 1. Fenchel showed 
that c(K) < dim if when the set of extreme points of K is connected [12]. Note that the 
Caratheodory number of an orbitope O v = conv(G • v ) in general depends on v (cf. [2"3"j 
Theorem 4.9]) whereas, for multiplicity-free representations, the dimension of O v does not. 

Question 3. What are the Caratheodory numbers of orbitopes? 

We refer to the recent work of Barvinok and Blekherman (U [7] for more information about 
the convex geometry of orbitopes and coorbitopes, especially with regard to their volumes. 

2.2. Algebraic geometry. Here we look at orbitopes as objects in real algebraic geometry. 
Fix a rational representation p : G — > GL(m,M.) of a compact connected algebraic group G. 
Every orbit G ■ v is an irreducible real algebraic variety in M m , and we may ask for its prime 
ideal. By the Tarski-Seidenberg Theorem [6j §2.4], the orbitope is a semi-algebraic set. 

Question 4. Which orbitopes are basic semi-algebraic sets, i.e. for which triples (G,p,v) 
can conv(G-f) be described by a finite conjunction of polynomial equations and inequalities? 

The boundary dO of an orbitope O in M. m is a compact semi-algebraic set of codimen- 
sion one in its affine span aff(O). The Zariski closure of dO is denoted by d a O. We call it 
the algebraic boundary of O. If aff(C) = M. m then the algebraic boundary d a O is the zero 
set of a unique (up to scaling) reduced polynomial f(xi,..., x m ) whose coefficients lie in the 
field of definition of (G,p,v). That field of definition will often be the rational numbers Q. 
Since scalars in Q have an exact representation in computer algebra, but scalars in M require 
numerical floating point approximations, we seek to use Q instead of K. wherever possible. 

Question 5. How to calculate the algebraic boundary d a O of an orbitope O? 

The irreducible factors of the polynomial f(x\, . . . , x m ) that cuts out d a O arise from 
various singularities in the boundary dO° of the coorbitope 0°. We believe that a complete 
answer to Question will involve a Whitney stratification of the real algebraic hypersurface 
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d a O° . Recall that a Whitney stratification is a decomposition into locally closed submanifolds 
(strata) in which the singularity type of each stratum is locally constant along the stratum. 
The faces of a polytope form a Whitney stratification of its boundary, which is dual to 
the stratification of the polar polytope. We expect a similar duality between the Whitney 
stratification of the boundary of an orbitope and of the boundary of its coorbitope. 

Question 6. How to compute and study the algebraic boundary d a O° of the coorbitope O ? 
Is every component of d a O the dual variety to a stratum in a Whitney stratification of d a O°? 

Recall that the dual variety X v of a subvariety X in R m is the Zariski closure of the set 
of all affine hyperplanes that are tangent to X at some regular point. When studying this 
duality, algebraic geometers usually work in complex projective space rather than real 
affine space lR m . In some of the examples for G = SO(n) seen in this paper, the algebraic 
boundary d a O° of the coorbitope O coincides with the dual variety X y of the orbit X = G-v. 
A good example for this is the discriminant al hypersurface in Corollary 16 .41 More generally, 
we have the impression that the hypersurface d a O° is often irreducible while d a O tends to be 
reducible. For further appearances of dual varieties in convex algebraic geometry see [23 

The k-th secant variety of G ■ v is the Zariski closure of all (fc+l)-flats spanned by points 
of G ■ v. The study of secant varieties leads to lower bounds for the Caratheodory number: 

Proposition 2.3. If k > c(O v ) then the k-th secant variety of G-v is the ambient space M. m . 

Proof. Let k > c(O v ). The set of points that lie in the convex hull of k + 1 points of G ■ v is 
dense in O v and hence is Zariski dense in R m . The fc-th secant variety contains that set. □ 

The lower bound for c(O v ) from Proposition 12.31 usually does not match Fenchel's upper 
bound c(O v ) < dim(O v ). For instance, consider the Caratheodory orbitope O v in Example 
11.11 Its algebraic boundary d a O v equals the second secant variety of the orbit G-v, so 
Proposition 12.31 implies c(O v ) > 3. This orbitope satisfies c{O v ) = 3 but dim(C„) = 4. 

Question 7. For which orbitopes O is the boundary d a O one of the secant varieties of G-v ? 
When is the lower bound on the Caratheodory number c(0) in Proposition \2. 3\ tight ? 

2.3. Optimization. A fundamental object in convex optimization is the set PSD n of posi- 
tive semidefinite symmetric real nxn-matrices. This is the closed basic semi-algebraic cone 
defined by the non-negativity of the 2 n — 1 principal minors. It can also be described by only 
n polynomial inequalities, namely, by requiring that the elementary symmetric polynomials 
in the eigenvalues, i.e. the suitably normalized coefficients of the characteristic polynomial, 
be non-negative. The algebraic boundary <9 a PSD n of the cone PSD ra is the symmetric nxn- 
determinant. All faces of PSD n are exposed, isomorphic to PSD& for k < n, and indexed by 
the lattice of linear subspaces ordered by reverse inclusion. 

Spectrahedra inherit these favorable properties. Recall that a spectrahedron is the inter- 
section of the cone PSD n with an affine-linear subspace in Sym 2 (lR n ). If we know that an 
orbitope is a spectrahedron then this either answers or simplifies many of our questions. 

Question 8. Characterize all SO (n) - orbitopes that are spectrahedra. 
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Polytopes are special cases of spectrahedra: they arise when the affine-linear space consists 
of diagonal matrices. One major distinction between polytopes and spectrahedra is that the 
class of spectrahedra is not closed under projection. That is, the image of a spectrahedron 
under a linear map is typically not a spectrahedron. See Section 5 for orbitopal examples. 
Characterizing projections of spectrahedra among all convex bodies is a major open problem 
in optimization theory; see e.g. [18]. Here is a special case of this general problem: 

Question 9. Is every orbitope the linear projection of a spectrahedron? 

In Questions Eland [HI it is important to keep track of the subfield of R over which the data 
(G, p, v) are defined. Frequently, this subfield is the rational numbers Q, and in this case we 
seek to write the orbitope as a (projected) spectrahedron over Q and not just over R. 

Semidefinite programming is the problem of maximizing a linear function over a (pro- 
jected) spectrahedron, and there are efficient numerical algorithms for solving this problem 
in practice. In our context of orbitopes, the optimization problem can be phrased as follows: 

Question 10. What method can be used for maximizing a linear function i over an orbitope 
O? Equivalently, how to evaluate the norm 1 1— > \\£\\o° associated with the coorbitope 0° ? 

This is equivalent to a non-linear optimization problem over the compact group G. We 
seek to find g G G which maximizes £{p{g) ■ v). This maximum is an algebraic function of v. 

3. Schur-Horn Orbitopes 

In this section we study two families of orbitopes for the orthogonal group G = 0(n). This 
group acts on the Lie algebra Ql n by restricting the adjoint representation of GL(n, R). The 
0(n)-module gl n decomposes into two distinguished invariant subspaces, namely Sym 2 R™ 
and A2R n . These correspond to the normal and tangent space of 0(n) C GL(n,M.) at the 
identity. In matrix terms, the spaces of symmetric and skew-symmetric matrices form two 
natural representations of 0(n) for the action g ■ A = gAg T with g G 0(n) and A G R nxn . 

For a symmetric matrix M G Sym 2 R n we define the symmetric Schur-Horn orbitope 

O m ■= conv(G • M) C Sym 2 R n . 
For a skew-symmetric matrix iV G A2R™ we define the skew-symmetric Schur-Horn orbitope 

O n := conv(G ■ N) C A 2 R n . 

We shall see that these orbitopes are intimately related to certain polytopes which govern 
their boundary structure and spectrahedral representation. This connection arises via the 
classical Schur-Horn theorem [33] . The material in the sections below could also be presented 
in symplectic language, using the moment maps of Atiyah-Guillemin-Sternberg [T] ITS]. 

3.1. Symmetric Schur-Horn orbitopes. The ) -dimensional space Sym 2 R n decom- 
poses into the trivial 0(n)-representation, given by multiples of the identity matrix, and 
the irreducible representation of symmetric nxn- matrices with trace zero. Every symmetric 
matrix M G Sym 2 R n is orthogonally diagonalizable over R. The ordered list of eigenvalues 
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of M is denoted A(M) = (Ai(M) > A 2 (M) > • • • > A„(M)). The orbit G • M equals the 
set of matrices A G Sym 2 IR n that satisfy \{A) = A(M). We shall see that its convex hull 
Om = conv(G • M) is the set of matrices A G Sym 2 IR n for which \{A) is majorized by A(M). 

For p,5 £ 1™ we say that q is majorized by p, written g < p, if gi + g 2 + • • • + Qn = 
Pi + T>2 + • • • + Pn, an d, after reordering, q x > • ■ ■ > q n and p\ > • • • > p n , we have 

<?i + <?2 H h <?fc < Pi + P2 H h Pfc for fc = 1, . . . , n - 1. 

For a fixed point p G M n , the set of all points q majorized by p is the convex polytope 

II(p) = {q E K n : q < p} = conv{vr ■ p = (p^i), . . . ,p 7r(n) ) : tt G ©„}. 

Here (5 n denotes the symmetric group, and H(p) is the permutahedron with respect to p. 
This is a well-studied polytope [29], [38] and is itself an orbitope for & n . The permutahedron 
n(p) for p = (pi > p 2 > • ■ ■ > Pn) consists of all points q G W 1 such that J2iPi = J2i Qi an( i 

\r\ 

(3.1) < j^Pi for all/ C [n]. 

iei i=l 

Let D : Sym 2 R™ — » M n be the linear projection onto the diagonal. 

Proposition 3.1 (The symmetric Schur-Horn Theorem [22]). Let M G Sym 2 IR n and Om 
its symmetric Schur-Horn orbitope. Then the diagonal D(M) is majorized by the vector of 
eigenvalues A(M). In fact, the orbitope Om maps linearly onto the permutahedron: 

D(O m ) = n(A(M)). 

Corollary 3.2. The Schur-Horn orbitope equals O m = {A G Sym 2 R™ : X(A) < A(M)}. 

Proof. We have shown that the right hand side equals {A G Sym 2 IR n : X(A) G D(Om)}- We 
claim that a matrix A is in this set if and only if A lies in Om- This is clear if A is a diagonal 
matrix. It follows for all matrices since both sets are invariant under the 0(n)-action. □ 

Our next goal is to derive a spectrahedral characterization of Om- Consider the natural 
action of the Lie group GL(n, M) on the k-th exterior power A^IR". If {v i, t>2, . . . , v n } is 
any basis of M™, then the induced basis vectors of A^IR™ are A Vi 2 A • • • A Vi k for 1 < 
i\ < i<i < ■ ■ ■ < ik < n. A matrix g G GL(n, M) acts on a basis element by sending it to 
g-v h Av i2 A---Av ik = (g ■ v h ) A (g -v i2 ) A • • • A (g -v ik ). We denote by C k : Ql(R n ) ->• sl(A fc M n ) 
the induced map of Lie algebras. The linear map is defined by the rule 

k 

(3.2) C k (B)(v h A f i 2 A • ■ ■ A f ifc ) = A • • • A v^_ x A (Su^) A A • • • A v lk . 

3=1 

The u) x (fc) -matrix that represents Ck(B) in the standard basis of AfcR n is known as the 
k-th additive compound matrix of the n x n-matrix B. It has the following main property: 

Lemma 3.3. Let B G Sym 2 IR n with eigenvalues \(B) = (Ai, A2, • • • , A n ). Then C k {B) is 
symmetric and has eigenvalues A^ + A, 2 + ■ — h A ifc for 1 < i\ < i 2 < ■ ■ ■ < i k < n. 
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Proof. Let vi, . . . , v n be an eigenbasis for B. Then the formula (13.21) says 

k 

C k (B)(v h A v i2 A • • • A v ik ) = 22 v h A • • • A A (A^.) A v ij+1 A • • • Av ik . 

i=i 

Hence t> ^ A t> j 2 A ■ ■ • A v ik is an eigenvector of Ck(B) with eigenvalue \ iL + ■ ■ ■ + X ik . □ 
This leads to the result that each symmetric Schur-Horn orbitope Om is a spectrahedron. 
Theorem 3.4. Let M G Sym 2 IR n with ordered eigenvalues X(M) = (Ai > • • • > A„). Then 

k 

O m = {AeSym 2 M n : Tr(A) = Tr(M) and ^AiId/»\-£ fc (A) y for k = 1, . . . , n-l}. 

i=i 

Proof. A matrix A e Sym 2 IR n is in Ojv/ if and only if X(A) is in the permutahedron I1(A(M)). 
From the inequality representation in ( 13. in . in conjunction with Lemma [3731 we see that this 
is the case if and only if the largest eigenvalue of X(Ck(A)) is at most Ai + h A/-. □ 

We shall now derive the description of all faces of the Schur-Horn orbitope Om- Since 
Om is a spectrahedron, by Theorem I3.4[ we know that all of its faces are exposed faces. 
Hence a face of Om is the set of points maximizing a linear function £ : Sym 2 IR n — > BL The 
canonical 0(n)-invariant inner product on Sym 2 M n is given by (A,B) = Tr(AB) and, by 
identifying spaces, a linear function may be written as £( ■ ) = (B, ■ ). Note that the dual 
space (Sym 2 R™)* is equipped with the contragredient action, that is, g ■ I — (g T Bg, ■ ). 

Theorem 3.5. Every 0(n)-orbit of faces of Om intersects the pullback of a unique & n - 
orbit of faces of the permutahedron n(A(M)). In particular, the faces of Om are products of 
symmetric Schur-Horn orbitopes of smaller dimensions corresponding to flags in R n . 

Proof. Let F be a face of Om and let i = {B, • ) be a linear function maximized at F . Then 
the face g ■ F is given by g ■ £ and we may identify 0(n) ■ F with the orbit 0(n) ■ £. Since B 
is orthogonally diagonalizable, the orbit 0(n) ■ £ contains the diagonal matrices 7r • X(B) for 
7i G & n . The corresponding faces are the pullbacks of the orbit of faces of n(A(M)) given by 
the linear function (A(£>), -)^n. Faces of the permutahedron correspond to flags of coordinate 
subspaces, and 0{n) translates these to arbitrary flags of subspaces. □ 

Facets of the permutahedron H(p) correspond to coordinate subspaces. Replacing these 
with arbitrary subspaces yields supporting hyperplanes for the Schur-Horn orbitope Om- 

Corollary 3.6. The Schur-Horn orbitope Om is the set of matrices A G Sym 2 R n such that 

Ty(A\l) < Tt(M\l) for every subspace L C M™. 

The following three examples show how Theorem 13.51 translates into explicit face lattices. 

Example 3.7 (The free spectrahedron). Let M = e\e\ G Sym 2 R n be the diagonal matrix 
with diagonal (1,0, ...,0). The orbitope Om is the convex hull of all symmetric rank 1 
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matrices with trace 1, and hence Om = PSD n R {trace = 1}. This orbitope plays the role of 
a "simplex among spectrahedra" because every compact spectrahedron is an affine section. 
The face O m of the orbitope Om in direction B e Sym 2 M n is isomorphic to 

conv{wM T : u E H Eig max (5)} 

where is the unit sphere and Eig max (5) is the eigenspace of B with maximal eigenvalue. 
Thus, O m is isomorphic to a lower dimensional Schur-Horn orbitope for a rank one matrix. 

We conclude that the face lattice of Om consists of the linear subspaces of 1R™ ordered by 
inclusion. This fact is well known; see [31 §11.12]. The dimension of a face corresponding to a 
fc-subspace is ( k ^) — 1. The projection D(Om) is the standard (n — l)-dimensional simplex 
A„_i = convjei, 62, ■ ■ ■ , e n } whose faces correspond to the coordinate subspaces. □ 

Example 3.8 (Spectrahedral hypersimplices) . We now describe continuous analogs to hy- 
persimplices, extending the simplices in Example 13. 71 Fix < k < n and let M € Sym 2 IR n be 
the diagonal matrix with k ones and n — k zeros. The orbitope Om of the (n, k)- spectrahedral 
hypersimplex, as its diagonal projection D(Om) = A(n, k) is the classical (n, k ) -hyper simplex; 
cf. [38J Example 0.11]. For instance, if n = 4 and k = 2 then Om is nine-dimensional and 
D(Om) is an octahedron. Up to ©^symmetry, the octahedron has one orbit of vertices and 
edges but two orbits of triangles. The pullback of any edge is a circle, and the pullbacks of 
the triangles are five- dimensional symmetric Schur-Horn orbitopes Om for A(M) = (1,0,0) 
and A(M) = (1, 1, 0). Both facets are isomorphic to free spectrahedra. □ 

Example 3.9 (The generic symmetric Schur-Horn orbitope). Let M G Sym 2 IR n be a sym- 
metric matrix with distinct eigenvalues, e.g. A(M) = (1,2, .. . ,n). The image of Om under 
the diagonal map is the classical permutahedron Yi n = n(l, 2,3, ... ,n). Its face lattice may 
be described as the collection of all flags of coordinate subspaces in M. n ordered by refinement. 

We may associate to every B e Sym 2 IR n the complete flag whose fc-th subspace is the direct 
sum of the eigenspaces of the first k largest eigenvalues. Thus, 0(n) ■ M may be identified 
with the complete flag variety over K. As for the facial structure, the face O m is isomorphic 
to the convex hull of the orbit stabo( n ){B) • M . Here, the stabilizer decomposes into a product 
of groups isomorphic to O(dj) where dj is the dimension of the z-th eigenspace of B. Hence, 
the face O m is isomorphic to a Cartesian product of generic Schur-Horn orbitopes and is 
of dimension ^ The face only depends on the flag associated to B. This implies 

that the face lattice of Om is isomorphic to the set of partial flags ordered by refinement. 
Again, in every orbit of flags there is a flag consisting only of coordinate subspaces. These 
special flags form the face lattice of the standard permutahedron n(l, 2, . . . , d) = D(Om)- 
We regard Om as a continuous analog of the permutahedron. □ 

We conclude this subsection with a discussion of the algebraic boundary 3JDm of the 
Schur-Horn orbitope. Let K be the smallest subfield of M that contains the eigenvalues 
Ai, . . . , A n , and suppose that the A, are sufficiently general. Then the hypersurface BJDm is 
defined in the affine space {A e Sym 2 (R") | Tr(A) = Tr(M)} by the following polynomial of 
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degree 2 n — 2 in ("'t ) unknowns over the field K: 



n-l 



f(A) = n det ( £fc(A) "S Ai - id ( 



k=l i=l 

However, from a computer algebra perspective, this is not what we want. Assuming that M 
has entries in Q, we prefer not to pass to the field extension IK, but we want the algebraic 
boundary d a Ou to be the Q-Zariski closure of the above hypersurface {f(A) = 0}. For 
instance, suppose that the characteristic polynomial of M is irreducible over Q. Then we 
must take the product of f(A) over all permutations of the eigenvalues Ai, . . . , A n , and the 
polynomial g(A) that defines d a Ou over Q is the reduced part of that product. It equals 

\n/2\ 

g{A) = H det(£ k (A)®C k {-M)), 

k=l 

where © denotes the tensor sum of two square matrices of the same size (see e.g. [28], §3]). 
Here the product goes only up to |"n/2] because the matrices A and M have the same trace. 

For special matrices M, the characteristic polynomial may factor over Q, and in this case 
the algebraic boundary d a Ou is cut out by a factor of the polynomial f(A) or g(A). 

3.2. Skew-symmetric Schur-Horn orbitopes. The space consists of skew-symmetric 
n x n-matrices N. The eigenvalues of N are purely imaginary, say ±zAi, . . . , ±iA&, where 
i = a/— 1, with k = [f J and an additional eigenvalue if n is odd. Thus iV is not diagonal- 
izable over R, but the adjoint 0(n)-action brings the matrix iV into the normal form 



gNg = y J ^ or n even an d gNg = | j for n odd 



Here g is a suitable matrix in 0(n), A is the diagonal matrix with diagonal Ai > ■ • ■ > A^ > 0, 
and we denote X(N) = (A x , A 2 , . . . , X k ). Let SD : A 2 lR n -»■ R k be the linear map such that 

SD(iV) = (JV M ,N 2tk+1 ,...,N ktn ) ifn = 2k, and 
SD(iV) = (N lik+1 ,N 2 , k +2,...,N ktn ) ifn = 2fc+l. 

If is in normal form as above, then SD(A^) = D(A) = A(AT). We call SD(A^) the skew- 
diagonal of N. In analogy to the symmetric case, the set SD(On) of all skew-diagonals 
arising from Om is nicely behaved; in fact, the necessary changes are rather modest. 

For a point q G M. k we denote by |g| = (\qi\, ■ ■ ■ , \q k \) the vector of absolute values. For 
p G M fc let U s (p) be the set of points q G M fc such that \q\ is weakly majorized by This 
means that |p| and |g| satisfy the majorization conditions except that J2i — 12i \Pi\ i s 
allowed. The polytope U s (P) is the B k -permutahedron. It is the convex hull of the orbit of p 
under the action of the Coxeter group B k , the group of all 2 k ■ k\ signed permutations. The 
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-Bfc-permutahedron II s (p) for p = (pi > P2 > ■ ■ ■ > Pk) consists of all points q G M fc with 

|/UJ| 

(3.3) ^2 <H ~ S q i - Yl Pi for any J ' J - ^ with 1 n J = ®- 

iei jeJ i=i 

As expected, we have the following analog of the symmetric Schur-Horn theorem. 

Proposition 3.10 (The skew-symmetric Schur-Horn theorem [22j). Let N e A2ffi n with 
X(N) = (Ai > • • • > Afc) and On the skew- symmetric Schur-Horn orbitope of N. Then 
|SD(iV)| is weakly majorized by (Ai, . . . , A^). In particular, we have SD(Ojv) = n s (A). 

The same arguments as in the symmetric case yield the following results. 

Theorem 3.11. Every 0(n) orbit of faces of the skew- symmetric Schur-Horn orbitope On 
contains the pullback of a unique B^-orbit of faces of the Bk-permutahedron I1 S (A(A^)). 

Corollary 3.12. The skew-symmetric Schur-Horn orbitope On coincides with the set of 
skew- symmetric matrices A such that |SD(A)| is weakly majorized by |SD(JV)|. 

Example 3.13. Fix n = 6 and k — 3, and p = (1,2,3). Then the system ( 13. 31) consists of 
26 linear inequalities, namely, six inequalities ig, < 3, twelve inequalities ±Oj ± gj < 5, and 
eight inequalities ±qi ± qj ± g fc < 6. Their solution set is the i? 3 -permutahedron IT(1, 2, 3), 
commonly known as the truncated cuboctahedron, and it has 48 vertices, 72 edges and 26 
facets (six octagons, twelve squares and eight hexagons). A picture is shown in Figure El 




Figure 2. The 5 3 -permutahedron is the truncated cuboctahedron. 

Let iV denote a general skew-symmetric 6x6-matrix. Theorem 13.111 implies that the facets 
of the 15-dimensional orbitope On come in three families, corresponding to 0(6)-orbits of 
the octagons, squares, and hexagons in Figure HJ The facets arising from the octagons are 
skew-symmetric Schur-Horn orbitopes for 50(4) with skew-diagonal (1, 2) and therefore have 
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dimension six. The facets arising from the squares are the product of a line segment and a 
disc, coming from 0(2) x 5*0(2) with 0(2) acting by the determinant. The facets arising 
from the hexagons are 0(3)-orbitopes isomorphic to symmetric Schur-Horn orbitopes with 
eigenvalues (1,2,3) and therefore have dimension five. □ 

We next present a spectrahedral description of an arbitrary skew-symmetric Schur-Horn 
orbitope On- To derive this, we return to symmetric matrices and their real eigenvalues. 

Lemma 3.14. Let N G A2IR™ be a matrix with eigenvalues ±zAi, . . . , ±i\k and let 

N = ( _® N Q ) E Sym 2 R 2 ". 

Then N has eigenvalues X±, Ai, A2, X 2 , . . . , A fc , A fc , — Ai, — Ai, — A2, — X 2 , • • • ? — Xk, —Xk- For any 
1 < 3 < k, the additive compound matrix C 2 j (N) has largest eigenvalue 2(Ai + A2 + • • • + Xj) ■ 

We conclude that each skew-symmetric Skew-Horn orbitope n s (A(iV)) is a spectrahedron: 

Theorem 3.15. Let N G A 2 M" with X(N) = (Ai > • • • > A fe ). Then 

O n = {Ae A 2 M" : 2(Ai + • • • + X^Id^nj - C 2j (A) h for 3 = 1, . . . , A;} . 

From this theorem we can derive a description of the algebraic boundary as before, and 
again the issue arises that the Xj lie in an extension IK over the field of definition of N, 
which will usually be Q. At present we do not know whether Theorems 13.41 and 13.151 can be 
extended to obtain spectrahedral representations of the respective orbitopes over Q. 

We close this section with one more example of a skew-symmetric Schur-Horn orbitope. 

Example 3.16. Consider the skew-symmetric Schur-Horn orbitope On for some N G A2M™ 
with X(N) = (1, 0, . . . , 0) G M fc . According to Theorem l3.15l a spectrahedral representation is 

O n = { A G A 2 R n : C 2 (A) < 2 ■ Id 2n }. 

The projection SD(On) to the skew diagonal is the crosspolytope conv{±ei, . . . , ie^}. 
This a regular polytope with symmetry group B^, and it has only one orbit of faces in each 
dimension. The orbitope On is the d = 2 instance of the Grassmann orbitope Qd, n - These are 
important in the theory of calibrated manifolds, and we shall study them in Section 7. □ 

4. Tautological Orbitopes 

We argued in Section [2] that, given a compact group G acting algebraically on V ~ IR n , we 
can identify G with a subgroup of 0(n), or even of SO(n) when G is connected. The ambient 
space End(V) ~ gl n is itself an n 2 -dimensional real representation of the group G. The action 
of G on End(V^) is by left multiplication. The orbit of the identity matrix Id n under this 
action is the group G itself. We call the corresponding orbitope conv(G) = conv(G • Id n ) 
the tautological orbitope for the pair (G, V). This orbitope lives in End(K), and it serves as 
an initial object because it maps linearly to every orbitope conv(G • v ) in V. Tautological 
orbitopes of finite permutation groups have been studied under the name of permutation 
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polytopes (see [29]). The most famous of them all is the Birkhoff polytope for G 
was studied for other Coxeter groups by McCarthy, Ogilvie, Zobin, and Zobin [25] . 

In this section we investigate the tautological orbitopes for the full groups O(n) and 50 (n). 
Similar to the Schur-Horn orbitopes in Section [3j the facial structure is governed by polytopes 
arising from the projection onto the diagonal. We begin with the example G = 50(3). 



4.1. Rotations in 3-dimensional space. The group 50(3) of 3x3 rotation matrices has 
dimension three. Its tautological orbitope is a convex body of dimension nine. The following 
spectrahedral representation was suggested to us by Pablo Parrilo. 

Proposition 4.1. The tautological orbitope conv(50(3)) is a spectrahedron whose boundary 
is a quartic hypersurface. In fact, a 3x3-matrix X = (x^) lies in conv(50(3)) if and only if 



(4.1) 



/ l+Xn+X 2 2+a;33 

Xi 3 - X 31 
X21 — X\2 



X32 — X23 
I+X11— £22— x 33 
X21 + Xl2 
Xl3 + ^31 



2^13 — ^31 
X21 + X\2 
-X11+X22— ^33 
2^32 + X23 



X21 — X\2 
X\3 + £31 
2^32 + %23 
-X\\— £22+2:33 / 

Proof. We first claim that conv(50(3)) coincides with the set of all 3x3-matrices 

2u 2 3 - 2Wi4 
M11-M22+M33-M44 
2«i2 + 2m 34 



y 0. 



(4.2) 



M11+M22-M33-M44 
2m 23 + 2wi4 

2w 24 - 2Mi 3 



2«i 3 + 2«24 

2u 34 - 2ui2 

U n -U 2 2~U 33 +U M/ 



where U 



[u 



ij 



runs over all positive semi definite 4x4- matrices having trace 1. 



Positive semi definite 4x4-matrices with both trace 1 and rank 1 are of the form 



U 





fa 2 


ab 


ac 


ad\ 


1 


ab 


b 2 


be 


bd 


a 2 + b 2 + c 2 + d 2 


ac 


be 


c 2 


cd 




\ad 


bd 


cd 


d 2 ) 



Their convex hull is the free spectrahedron of Example 13.71 The image of the above rank 1 
matrices U under the linear map (14. 2 j) is precisely the group 50(3). This parametrization is 
known as the Cayley transform. Geometrically, it corresponds to the double cover SU (2) — > 
50(3). The claim follows because the linear map commutes with taking the convex hull. 



The symmetric 4x4- matrices U 



[Ui 



with trace(i7) = 1 form a nine- dimensional affine 



space, and this space is isomorphic to the nine- dimensional space of all 3x3-matrices X = 
(x^) under the linear map given in (14. 2p . We can express each Uij in terms of the x^i by 
inverting the linear relations x\\ = u\\ + U22 ~u 33 — m 44 , xyi = 2u 2 3 — 2ui 4 , etc. The resulting 
symmetric 4x4- matrix U is precisely the matrix (14.11) in the statement of Proposition ^. 11 □ 

The ideal of the group 0(3) is generated by the entries of the 3x3-matrix X-X T —ld 3 , while 
the prime ideal of 50(3) is that same ideal plus (det(X) — 1). We can check that the prime 
ideal of 50(3) coincides with the ideal generated by the 2x2- minors of the matrix (14. ip . 
Thus the group 50(3) is recovered as the set of matrices (14. ip of rank one. 



ORBITOPES 



17 



Proposition 14.11 implies that conv(S'0(3)) is affinely isomorphic to the free spectrahedron 
for n = 4, that is, to the set of positive semidefmite 4x4- matrices with trace equal to 1. 
This implies a characterization of all faces of the tautological orbitope for 50(3). First, all 
faces are exposed because conv(S'0(3)) is a spectrahedron. All of its proper faces are free 
spectrahedra, for n = 1, 2, 3, so they have dimensions 0, 2 and 5, as seen in Example 13.71 

4.2. The orthogonal group. We now examine 0(n) = {X G M. nxn : X ■ X T = Id n }. As 
before, let D : M. nxn — > R™ denote the projection of the n x n- matrices onto their diagonals. 

Lemma 4.2. The projection D(conv(0(n)) of the tautological orbitope for the orthogonal 
group 0(n) to its diagonal is precisely the n- dimensional cube [—1, +l] n . 

Proof. The columns of a matrix X G 0(n) are unit vectors. Thus every coordinate Xij in 
bounded by 1 in absolute value and D(conv(0(n)) is a subset of the cube. For the reverse 
inclusion, note that all 2 n diagonal matrices with entries ±1 are orthogonal matrices. □ 

The cube [— l,+l] n is the special 5 n -permutahedron IF(1, 1, . . . , 1). As with Schur- 
Horn orbitopes, the projection onto this polytope reveals the facial structure. As gen- 
eral endomorphisms are not normal, the key concept of diagonalizability is replaced by 
that of singular value decomposition. Recall that for any linear map A G W axn there 
are orthogonal transformations U, V G 0(n) such that UAV T is diagonal with entries 
o~(A) = (o- 1 (A)>a 2 {A)> ■ ■ ■ >o- n (A)) G K> . These entries are the singular values of A. 

We shall see in (14.41) that conv(0(n)) is a spectrahedron, hence all of its faces are exposed 
faces. The following result recursively characterizes all faces of this tautological orbitope. 

Theorem 4.3. Let £ = (B, ■) be a linear function on M. nxn with B G M. nxn . Then the face 
of conv(0(n)) in direction £ is isomorphic to conv(0(m)) where m = dimker(fi). 

Proof. Let £(■) = (B, ■ ) be a linear function with B G R nxn , so that £(A) = Trace(AB). We 
fix a singular value decomposition UHV = B of the matrix B. Here E is a diagonal matrix 
nxn with its first n — m entries positive and remaining m entries zero. This matrix also 
defines a linear function £'(■) = (£, ■ ) on IR nxn . Cyclic invariance of the trace ensures that 
the faces conv(0(n)) £ and conv(0(n))^ are isomorphic. The subset of 0(n) at which £' is 
maximized is the subgroup {g G 0(n) : g ■ Ci = e« for i = 1, . . . , n — m}. The convex hull of 
this subgroup equals conv(0(n)) £ . It coincides with the tautological orbitope for 0(m). □ 

We interpret Theorem 14.31 geometrically as saying that the tautological orbitope for 0(n) 
is a continuous analog of the n-dimensional cube. Every face of the cube is a smaller 
dimensional cube and the dimension of a face maximizing a linear functional £ is determined 
by the support of £. The role of the support is now played by the rank of the matrix B. This 
behavior yields information about the Caratheodory number of the tautological orbitope. 

Proposition 4.4. The Caratheodory number of the orbitope conv(0(n)) is at most n+1. 

Proof. By [2"3"| Lemma 3.2], the Caratheodory number of a convex body K is bounded via 

< 1 + max{c(F) : F C K a proper face }. 
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Since every proper face is isomorphic to conv(0(fc)) for some k < n the result follows by 
induction on n. The base case is n = 1 for which conv(0(l)) is a 1-simplex. □ 

Note that the orbit 0(n) ■ Id n coincides with the orbit of the identity matrix Id n under the 
action of the product group 0(n) x 0(n) by both right and left multiplication. Hence the 
tautological orbitope conv(0(n)) is also an 0(n) x 0(n)-orbitope for that action. We shall 
now digress and study these orbitopes in general. After we have seen (in Theorem 14.7ft that 
these are spectrahedra, we shall resume our discussion of conv(0(n)). 

4.3. Fan orbitopes. The group G = 0(n) x 0(n) acts on R nxri by simultaneous left and 
right translation. The action is given, for (g, h) G 0(n) x 0(n) and A G M nx?1 , by 

(4.3) (9,h)-A := gAh T . 

Ky Fan proved in [TU] that the Schur-Horn theorem for symmetric matrices under conjugation 
by 0(n) generalizes to arbitrary square matrices under this 0(n)xO(n) action. Now, singular 
values play the role of the eigenvalues. The following is a convex geometric reformulation: 

Lemma 4.5 (Ky Fan [10]). For a square matrix A G IR nxn let Oa denote its orbitope under 
the action ft4.3\ ) of the group 0(n) x 0(n). Then the image D(Oa) of the projection to the 
diagonal is the B n -permutahedron with respect to the singular values o~(A). 

We shall refer to Oa = conv{(<?, h) ■ A : g,h G 0(n)} as the Fan orbitope of the matrix 
A. From Lemma I4.5[ one easily deduces the analogous results to Theorems 13.51 and 13.111 

Remark 4.6. The facial structure of the Fan orbitope Oa is determined by the facial 
structure of the S n -permutahedron U s (a(A)) specified by the singular values of the matrix A. 

The description of the 5 n -permutahedron in terms of weak majorization was stated 
in (13.31) . Rephrasing these same linear inequalities for the singular values, and using Lemma 1475 
now leads to a spectrahedral description of the Fan orbitopes. For that we make use of the 
alternative characterization of singular values as the square roots of the (non-negative) eigen- 
values of AA T . Using Schur complements, it can be seen that the 2n x 2n-matrix 




has the eigenvalues ±ai(A), . . . , ±o~ n (A). We form its additive compound matrices as before. 

Theorem 4.7. Let A be a real nxn-matrix with singular values o~\ > o~2 > • • • > o~ n . Then 
its Fan orbitope Oa equals the spectrahedron 

O a = {XeR nxn : C k {S{X)) H ( ai + ... + (Tfe )Id^ k = l,...,n}. 

Fix an integer p G {1,2, ... ,n}. The Ky Fan p-norm is defined by 

v 

X ^ 5>*(x). 

i=i 
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This function is indeed a norm on M nxri , and its unit ball is the Fan orbitope Oa where A 
is the diagonal matrix with p diagonal entries 1/p and n — p diagonal entries 0. Theorem 
14.71 shows that the unit ball in the Ky Fan p-norm is a spectrahedron. Two norms of special 
interest in applied linear algebra are the operator norm \\A\\ := o~i(A) and the nuclear norm 
\\A\\* := + • • • + o~ n (A). Indeed, these two norms played the key role in work of Fazel, 

Recht and Parrilo [31] on compressed sensing in the matrix setting. Both the operator norm 
and the nuclear norm are closely tied to their vector counterparts. 

Remark 4.8. The unit balls in the operator and nuclear norm are both Fan orbitopes. The 
projection to the diagonal yields the unit balls for the and the ^-norm on MJ 1 . These two 
unit balls are polytopes in M n , namely, the n-cube and the n-crosspolytope respectively. 

Fazel et al. showed in [3TJ Prop. 2.1] that the nuclear norm ball has the structure of a 
spectrahedron, and semidefmite programming duality allows for linear optimization over the 
operator norm ball. The spectrahedral descriptions of both unit balls in M. nxn are special 
instances of Theorem 14. 7\ to be stated explicitly once more. The operator norm ball consists 
of all matrices X whose largest singular value is at most 1. This is equivalent to 



The nuclear norm ball consists of all matrices X for which the sum of the singular values 
is at most 1. This is equivalent to saying that the sum of the largest n eigenvalues of the 



We are now finally prepared to return to the main aim of this section, which is the study 
of tautological orbitopes. The proof of Theorem 14.31 implies that the convex hull of 0(n) 
equals the set of nxn- matrices whose largest singular value is at most 1. In other words: 

Corollary 4.9. The operator norm ball in M nxn is equal to the tautological orbitope of the 
orthogonal group 0(n). The coorbitope conv(0(n))° is the nuclear norm ball. Both of these 
convex bodies are spectrahedra. They are characterized in jjj4-4\ ) an d i4-5\ ) respectively. 

It would be worthwhile to explore implications of our geometric explorations of orbitopes 
for algorithmic applications in the sciences and engineering, such as those proposed in [3 1 j . 

4.4. The special orthogonal group. We next discuss the faces of the tautological orbitope 
of the group SO(n). The relevant convex polytope is now the convex hull HC n of all vertices 
v G { — 1, +1}" with an even number of (— l)-entries. This polytope is known as the demicube 
or half cube. It is a permutahedron for the Coxeter group of type D n , i.e. signed permutations 
with an even number of sign changes. The facet hyperplanes of HC n are derived by separating 



(4.4) 





nxn 



(4.5) 



Id/2nN " C n (S(X)) h 0. 
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infeasible vertices of [—1, +1]™ by hyperplanes through the n neighboring vertices: 

(4.6) HC n = {x e [-l,+l] n : ^Xi-^Xi < n-2 for all J C [n] of odd cardinality}. 

While HC 2 is only a line segment, we have dim(HC n ) = n for n > 3. For example, the halfcube 
HC3 is the tetrahedron with vertices (1, 1, 1), (—1, —1, 1), (—1, 1, —1) and (1, —1, —1). Note 
that its facet inequalities appear as the diagonal entries in the symmetric 4x4- matrix ( |4.ip : 

HC 3 = {16R 3 : min(l+Xi+:r 2 +£3, l+x% — x 2 — x 3 , 1— Xi+x 2 — x 3 , 1— X\— x 2 +x 3 ) > }. 

This observation is explained by results of Horn (cf. [22]) on the diagonals of special orthog- 
onal matrices. These imply the following lemma about the tautological orbitope of SO(n): 

Lemma 4.10. The projection of conv(SO(n)) onto the diagonal equals the halfcube HC n . 

Proof. Just like in Lemma I4.2[ it is clear that HC„ is a subset of D(conv(S'0(n)). The 
converse is derived from the linear algebra fact that the trace of any matrix in 0(n)\SO(n) 
is at most n — 2. For J C [n] let Rj be the diagonal matrix with (Rj)u = — 1 if i G J and 
(Rj)u — 1 if i $l J. Let g G SO(n). Then trace(g • Rj) < n — 2 for all J of odd cardinality. 
This means that D(g) satisfies the linear inequalities in (14. 6[) and hence lies in HC„. □ 

There is a variant of singular value decomposition with respect to the restricted class 
of orientation preserving transformations. For every matrix A G R nx " there exist rotations 
U, V G SO(n) such that UAV is diagonal. The diagonal entries are called the special singular 
values and denoted by &(A) = (ai(A) > ••• > a n (A)). The main difference to the usual 
singular values is that cr n (A) may be negative; only the first n — 1 entries of 5(A) are non- 
negative. We need to make this distinction in order to understand the faces of conv(S'0(n)). 

Theorem 4.11. The tautological orbitope of SO(n) has precisely two orbits of facets. These 
are the tautological orbitopes for SO{n — 1) and the free spectrahedra of dimension Q) — 1- 

Proof. Up to -D^-symmetry, the halfcube HC n has only two distinct facets, namely an [n — 1)- 
dimensional halfcube and an (n — l)-dimensional simplex. A typical halfcube facet (HC n )^ 
arises by maximizing the linear function i = x\ over HC n , and a typical simplex facet (HC n ) £ 
arises by maximizing the linear function £' = xi+x 2 + - ■ — x n . Pulling back i along the 

diagonal projection D, we see that the facet of conv(S , 0(n)) corresponding to the halfcube 
facet is the convex hull of all rotations g G SO(n) that fix the first standard basis vector e\. 
Pulling back £' along D, we see that the facet of conv(SO(n)) corresponding to the simplex 
facet is the convex hull of {(? G SO{n) : Tr(g • R{ n }) = n — 2}. This facet is isomorphic 
to the convex hull of all g' G 0(n)\SO(n) such that Tr(g') = n — 2. Since g' is orientation 
reversing, one eigenvalue of g' is —1, and Tr(g') = n — 2 forces all other eigenvalues to be 
equal to 1. Hence the facet in question is the symmetric Schur-Horn orbitope for the diagonal 
matrix (1, . . . , 1, — 1). Example 13.71 implies that this is a free spectrahedron. □ 

In Subsection 4.1 we exhibited a spectrahedral representation for the tautological orbitope 
conv(S'0(3)), and in that case, the two facet types of Theorem 14.111 collapse into one type. 
At present, we do not know how to generalize the representation (14.1 p to SO(n) for n > 4. 
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5. Caratheodory Orbitopes 

Orbitopes for 5*0(2) were first studied by Caratheodory [Sj. The coorbitope cone (I2.3P dual 
to such a Caratheodory orbitope consists of non-negative trigonometric polynomials. This 
leads to the Toeplitz spectrahedral representation of the universal Caratheodory orbitope in 
Theorem l5.2t which implies that the convex hulls of all trigonometric curves are projections of 
spectrahedra. The universal Caratheodory orbitope is also afhnely isomorphic to the convex 
hull of the compact even moment curve, whose coorbitope cone consists of non-negative 
univariate polynomials. This leads to the representation by Hankel matrices in Theorem 15.61 

5.1. Toeplitz representation. The irreducible representations p a of S0(2) are indexed by 
non-negative integers a. Here, p is the trivial representation. When a 6 Nis positive, the 
representation p a of 5*0(2) acts on M 2 , and it sends a rotation matrix to its ath power: 

/cos(0) -sin(0)\ / cos(0) -sin(0) V _ (cos(a6) - sin(a0) \ 

pa : \sm{0) cos(0) J ^ \ sin(0) cos(0) J ~ \sin(a6) cos(a0) J ' 

For a vector A = (a 1; a 2 , . . . , a^) G N d we consider the direct sum of these representations 

Pa ■= p ai © Pa 2 © • ■ ■ © Pa d - 

The Caratheodory orbitope Ca is the convex hull of the orbit 50(2) ■ (1, 0) d under the action 
Pa on the vector space (IR 2 )^. This orbit is the trigonometric moment curve 

(5.1) | ( cos(M), trin(ai0), . . . , cos(a d 0), sin(a d 0) ) G R 2d : 9 G [0, 2tt) }. 

This curve is also identified with the matrix group p^(50(2)) lying in the space (M 2x2 ) d of 
block-diagonal 2d x 2d- matrices with d blocks of size 2x2. Thus Ca is isomorphic to the 
convex hull of pa(SO(2)), and can therefore also be thought of as a tautological orbitope. 

We distinguish between isomorphisms of orbitopes that preserve the 50 (2)-action, and 
the weaker notion of affine isomorphisms that preserve their structure as convex bodies. 

Lemma 5.1. Any orbitope of the circle group 50(2) is isomorphic to a Caratheodory 
orbitope Ca, where A G N d has distinct coordinates, and it is affinely isomorphic to a 
Caratheodory orbitope where the coordinates of A are relatively prime integers. 

Proof. Let O be an orbitope for 50(2). We may assume that its ambient 50 (2)-module 
V has no trivial components and is the linear span of O. Then V has the form pa for 
A G N d with distinct non-zero components, that is, V is multiplicity free. This is because 
Endso(2)(Pa) = C, where we identify M 2 with C and 50(2) with the unit circle in C. The 
orbitope O is generated by a vector v = (vi,...,Vd) G C d with non-zero coordinates. By 
complex rescaling, we may assume that v = (1, . . . ,1), showing that O is isomorphic to Ca- 
Lastly, if the coordinates of A = (a 1; . . . , ad) have greatest common divisor a, then pa is the 
composition pa> ° p a , where A' = (a\/a, . . . , ad/a) and O is affinely isomorphic to an orbitope 
for the module pa>- □ 
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We henceforth assume that < a% < ■ ■ ■ < ad where the ai are relatively prime. When 
A = (1, 2, . . . , d), Caratheodory [8| studied the facial structure of Ca- An even- dimensional 
cyclic polytope is the convex hull of finitely many points on Caratheodory's curve (see |38j). 
Smilansky [M] studied the four- dimensional Caratheodory orbitopes (d = 2), and this was 
recently extended by Barvinok and Novik [5] who studied C(i,3,5,....2fc-i)- The corresponding 
curve (15. 1ft is the symmetric moment curve which gives rise to a remarkable family of centrally 
symmetric polytopes with extremal face numbers. Many questions remain about the facial 
structure of the Barvinok- Novik orbitopes C(i,3,5,...,2fc-i)- See §7.4] for details. 

We now focus on the universal Caratheodory orbitope Cd '■— C(i,2,. ..,<*) in This convex 
body has the following spectrahedral representation in terms of Hermitian Toeplitz matrices. 



Theorem 5.2. The universal Caratheodory orbitope Cd is isomorphic to the spectrahedron 
consisting of positive semidefinite Hermitian Toeplitz matrices with ones along the diagonal: 

I 1 xi ■■■ Xd-l Xd \ 



(5.2) 



Vd-i 
Vd 



Vd- 
Vd- 



X d -2 Xd-1 



1 

in 



X\ 

1 



y o 



where 



Xj 

y.i 



+ s 




J 



We note that the complex spectrahedron (15.21) can be translated into a spectrahedron over 
K. as follows. Consider a Hermitian matrix H = F + G ■ i where F is real symmetric and G 
is real skew-symmetric. The Hermitian matrix H is positive definite if and only if 

F -G 
G F 



y o. 



A trigonometric curve is any curve in W 1 that is parametrized by polynomials in the 
trigonometric functions sine and cosine, or equivalently, any curve that is the image under a 
linear map of the universal trigonometric moment curve (15. ip where A = (1, 2, . . . , d). 

Corollary 5.3 (cf. Henrion [19]). The convex hull of any trigonometric curve is a projected 
spectrahedron. In particular, all Caratheodory orbitopes are projected spectrahedra. 

Figure [3] shows the convex hulls of three trigonometric curves in M. 3 . The left and middle 
convex bodies are each the intersection of two convex quadratic cylinders (2x 2 — 1 + z and 



z for the former; 



x 2 + y 2 



1 and 2z = 1 + x for the latter) and hence are 



spectrahedra. The rightmost convex body is visibly not a spectrahedron. Its exposed points 
are (cos(#), sin(2#), cos(3#)) for 9 G (— |, |)U(y : , 4^), and it has two algebraic families of one- 
dimensional facets. In addition, there are two two-dimensional facets, namely the equilateral 
triangles for 9 — 0, ^> if an d 9 = tt, |, — |. The six edges of these two triangles are one- 
dimensional non-exposed faces, as there is no linear function which achieves its minimum on 
this body along these edges. Also, exactly one vertex of each triangle at 9 = and 9 = tt 
is exposed. We conclude that this convex body is a projected spectrahedron but it is not a 
spectrahedron. 
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(cos(0), sin(0), cos(2#)) (cos(20), sin(20), cos(0)) (cos(0), sin(20), cos(30)) 
Figure 3. Convex hulls of three trigonometric curves. 

Proof of Theorem \5.2 . The coorbitope cone dual to Cd consists of all affine- linear functions 
that are non-negative on Cd- These correspond to non-negative trigonometric polynomials: 

d 

C° d = {(5,a l ,b 1 ,...,a d ,b d ) G R 2d+1 : 5 + a k cos(k6) + b k sm(k6) > for all 0}. 

k=i 

We identify each point (5, ax, b\, . . . , a d , b d ) in R 2a!+1 with the Laurent polynomial 

d 

(5.3) R(z) = u *** e ^l^z- 1 ] 

k=-d 

with uq = 5, Uk = \(dk — bki), i = V~ 1, and U-k = Uk- Then R(z) = R{z^) and R G C d if 
and only if R is non-negative on the unit circle S 1 of C. Roots of R occur in pairs a, a -1 and 
those on S 1 have even multiplicity. Choosing one root from each pair gives the factorization 

(5.4) R(z) = H{z~ 1 )-H{z), 

where H G C[z] has degree d and the coefficient vectors of H and H are complex conjugate. 
This factorization is the classical Fejer-Riesz Theorem. 

Utilizing the monomial map 7^ : C — > C d+1 with jdi 2 ) — (1 ; z,z 2 , . . . , z d ) T , this is equiva- 
lent to the following: A trigonometric polynomial R{z) is non-negative on the unit circle if 
and only if there is a non-zero vector h G C d+1 such that 

(5.5) R(z) = id{z~ l ) T -hh T - ld (z). 

A point (ci, si, . . . , c d , Sd) G M 2d belongs to the Caratheodory orbitope Cd if and only if 

d 

(5.6) S + ^2 a k°k + bkSk > for all (5, a 1} b u . . . , a d , b d ) G C° d . 

fc=i 

The sum on the left equals the Hermitian inner product in C 2d+1 of the coefficient vector u 
of the polynomial R{z) and the vector ( = (x,l,y) with Xk, yu as in (15. 2ft . The formula (15. 5ft 
expresses u as the image of the Hermitian matrix hh T under some linear projection n. If ti* 
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denotes the linear map dual to 7r then X = tt*{() is precisely the Hermitian Toeplitz matrix 
in (15.21) . We conclude that the sum in (I5.6P equals 

(C,vr(M T )) = (n*{(),hh T ) = Ti{X-hh T ) = h T ■ X -h. 

Thus the point (ci, sx, ■ ■ ■ , Q, Sd) represented by a Hermitian Toeplitz matrix X lies in Cd if 
and only if h T ■ X ■ h > for all h G C d+1 if and only if X is positive semidefinite. □ 

The proof of Theorem 15.21 elucidates the known results about the facial structure of Cd- 

Corollary 5.4. The universal Caratheodory orbitope Cd is a neighborly simplicial convex 
body. Its faces are in inclusion-preserving correspondence with sets of at most d points on 
the circle. 

Proof. A Laurent polynomial R as in ( 15. 3 p lies in the boundary of the coorbitope cone C° d 
if and only if it is non-negative on the unit circle S 1 but not strictly positive. It supports 
the face of Cd spanned by the points of the trigonometric moment curve corresponding to 
its zeros in S 1 . Each zero has multiplicity at least 2, so there are at most d such points, and 
conversely any subset of < d points supports a face. Since any fewer than 2d +2 points on the 
curve are affinely independent and since all faces are exposed, these faces are simplices. □ 

Corollary 15.41 implies that the Caratheodory number c(Cd) equals d+1, as no point in the 
interior of Cd lies in the convex hull of d points of the orbit, but c(Cd) is at most one more 
than the maximal Caratheodory number of a facet, by Lemma 3.2 of |23j . 

Caratheodory orbitopes are generally not spectrahedra because they can possess non- 
exposed faces. Smilansky [31] showed that if we write p{9) e R 4 for a point on the trigono- 
metric moment curve with weights 1 and 3 then the faces of Cn,3) are exactly the points p(8) 
of the orbit, the line segments conv{p(#), p{6 + a)}, where < a < and the triangles 

conv{p{6), p{6 + 2tt/3), p(fl + 4vr/3)}. 

In particular, each line segment conv{p(0), p{9 + ^} is a non-exposed edge of C(i,3). We 
conclude that the Barvinok-Novik orbitope C(i,3) is not a spectrahedron. 

The Toeplitz representation ( 15. 2ft of the universal Caratheodory orbitope Cd reveals com- 
plete algebraic information. For example, the algebraic boundary d a Cd is the irreducible 
hypersurface of degree d + 1 defined by the determinant of that {d+1) x (d+l)-matrix. The 
curve ( 15. ip itself is the set of all positive definite Hermitian Toeplitz matrices of rank one. 
The 2 x 2-minors of the matrix ( 15. 2ft generate the prime ideal Ja,...,d) of this rational curve. 

The union of the {j — l)-dimensional faces of Cd is the set of positive definite Hermitian 
Toeplitz matrices of rank point lies on a {j — l)-dimensional face if and only if it 

is the convex combination of j points of the curve. The Zariski closure of this stratum is 
the set of all rank j Hermitian Toeplitz matrices which is defined by the vanishing of the 
{j+1) x {j+1) minors of that matrix. This is also the jth secant variety of the Caratheodory 
curve. Lastly, this rank stratification is a Whitney stratification of the algebraic boundary. 

The derivation of the algebraic description of Ca for arbitrary A = {ax, a 2 , . . . , a^) requires 
the process of elimination. For instance, the ideal J a of the trigonometric moment curve 
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(15. ip can be computed from the ideal of 2x2-minors for J(\^,...,a d ) by eliminating all unknowns 
Xj, yj with j G" {ai, . . . , ad}- The equation of the algebraic boundary d a CA is obtained by the 
same elimination applied to a certain ideal of larger minors of the Toeplitz matrix (15. 2p . 

We refer to recent work of Vinzant [36] for a detailed study of the edges of Barvinok-Novik 
orbitopes. An analysis of the algebraic boundary the orbitope C(i,3) can be found in [301 §2.4]. 

5.2. Hankel representation. The cone over the degree d moment curve is the image of M 2 
in its dth. symmetric power Sym d IR 2 ~ R d+1 under the map 

v d : (x,y) ^ (x\x d - x y,x d - 2 y\ ...,/). 

This map is naturally SO (2)-equivariant. We define the compact moment curve to be the 
image ^(S 1 ) of the unit circle under the map v d . This restricted map equals 

S 1 3 6 I—)- (cos^^os^^sin^),...,^^)). 

The convex hull of the curve ^(S 1 ) is an orbitope. By Lemma [5.11 it is isomorphic to some 
Caratheodory orbitope Ca- The following lemma makes this identification explicit. 

Lemma 5.5. If d eNis odd, then conv(z/rf(S 1 )) is isomorphic to the Barvinok-Novik orbitope 
C(i,3,...,d)- If d G N is even, then conv(^(§ 1 )) is isomorphic to C(o,2,4,...,d); which is affinely 
isomorphic to the universal Caratheodory orbitope Cd/2- 

Proof. Complexifying the S'0(2)-module pA where A = (ai, . . . ,a d ) gives the C x -module 
with symmetric weights ±a 1; . . . , ±a^. Thus the underlying real SO (2)-module of this C x - 
module is P([ai|,...,|a d |)- The lemma follows because the complexified representation Sym d C 2 
of Sym d lR 2 has weights d, d—2, d—4, . . . , — d, and this representation is spanned by the pure 
powers of linear forms, so every weight appears in the linear span of the orbit. □ 

Suppose now that d = 2n is even. We describe the moment curve and the Caratheodory 
orbitope in coordinates (Ao, Ai, . . . , A2 ra ) for M 2n+1 . Fix the (n+1) x (n+l)-Hankel matrix 







/Ao 


Ai 


A 2 • 


• A n \ 






Ai 


A 2 


A 3 • 


A ra+ i 


(5.7) 


K(X) = 


A 2 


A 3 


A 4 • 


A n+ 2 






VAn 


A n +i 


A n +2 ' 


A 2n / 



Theorem 5.6. The even moment curve consists of all vectors A G R such that the 
Hankel matrix K(X) has rank one, is positive semidefinite, and satisfies the linear equation 

(5.8) = !• 

Its convex hull consists of all A such that K(X) is positive semidefinite and satisfies (15. 8p . 
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Thus the Caratheodory orbitope C n has a second Hankel representation as a spectrahe- 
dron. The proof of this theorem follows from the well-known fact that every non-negative 
polynomial in one variable is a sum of squares of polynomials. It uses the duality in [32, §3]. 

Proof. Observe that the points of the even compact moment curve satisfy ( 15.81) . which comes 
from the polynomial identity (x 2 + y 2 ) n = 1. As Sym 2n M 2 has just one copy of the trivial 
representation, this is the only affine equation that holds on ^(S 1 ). The dual to Sym 2n M 2 is 
the space M[x, y\ 2n of real homogeneous polynomials of degree In in x and y. The coefficients 
of such a polynomial gives coordinates for R.[x, y] 2n - 

The coorbitope cone ^(S 1 ) dual to the orbitope conv(z/ 2n (S 1 )) is the cone of homogeneous 
polynomials of degree 2n that are non-negative on M 2 . Thus a point (Ao, . . . , A2 n ) £ Sym 2n IR 2 
lies in the orbitope conv(t / 2n (§ 1 )) if and only if it satisfies ( 15.8]) and 

2n 

(5.9) J^f^ ^ 

i=0 

for every non-negative polynomial f(x,y) = 'Y^ i fiX l y 2n ~ t . 

Since non-negative homogeneous polynomials in x and y are sums of squares (cf. |32j), 
we only need these inequalities to hold when f(x,y) = g(x,y) 2 is a square. Writing g = 
(go, . . . , g n ) for the coefficient vector of the polynomial g(x, y), the sum (I5.9P becomes 

2n 

i=0 j+k=i 

where K(X) is the Hankel matrix (15. 7\i . This proves the theorem. □ 

6. Veronese Orbitopes 

The Hankel representation of the universal Caratheodory orbitope arose by considering 
the image of the circle S 1 C M 2 in Sym 2n R 2 and its relation to non-negative binary forms. 
Generalizing from IR 2 to M. d gives the Veronese orbitopes whose coorbitope cones (I2.3P consist 
of non-negative d-ary forms. When d > 3, non-negative forms are not necessarily sums of 
squares, except for quadratic forms and the exceptional case of ternary quartics. 

The set of decomposable symmetric tensors is the image of the Veronese map 

j j ( d-\-m — l\ 

v m : R d — ► Sym m R d ~ B\ > . 

The SO(d)-oxb\ts through any two non-zero decomposable tensors are scalar multiples of 
each other and are thus isomorphic. We define the Veronese orbitope Vd, m to be the convex 
hull of the orbit through the specific decomposable tensor v m (l, 0, . . . , 0). That orbit is also 
the image z/ m (S d_1 ) of the unit (d — l)-sphere under the m-th Veronese embedding of lR d . 

Suppose that m = 2n is even. Then the orbit i > rn (Ei d ~ 1 ) can be identified with RP d_1 since 
v 2n is two-to-one with V2n(v) = v 2n (—v). The dual vector space to Sym 2n M d is the space 
of homogeneous forms of degree 2n on M d . The only invariant forms are those proportional 
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to the form (v,v) n , so both Sym 2n R d and its dual space contain one copy of the trivial 
representation, and ^2n(§> d_1 ) lies in the hyperplane of Sym 2n IR d defined by (v,v) n = 1. 

The dual cone to the Veronese orbitope Vd,2n = conv(t/ 2n (S d_1 )) is the cone of non- 
negative forms of degree 2n in Sym 2n (IR d )*. See also jU Example (1.2)]. We write V^ 2n 

for the Veronese coorbitope cone consisting of non- negative forms. The cone 2n of non- 
negative forms contains the cone K^n of sums of squares, but when d > 3, 2n > 4, and 
(d,2n) 7^ (3,4), Hilbert [20] showed that the inclusion is strict. We refer to [7] for a recent 
study which compares the dimension of the faces of these cones. 

The cone /Q j2n is naturally a projection of the positive semidefinite cone. Hence its dual 
cone Crf j2n = K? d 2n is a spectrahedron: it can be realized as the intersection of the positive 
semidefinite cone with a certain linear space of generalized Hankel matrices, discussed in 
detail in Reznick's book [52] • This spectrahedron Cd^n is strictly larger than the Veronese 
orbitope Vd,2n when d > 3, 2n > 4, and (d, 2n) ^ (3, 4). In fact, Vd,2n is precisely the convex 
hull of the subset of extreme points in Cd,2n that have rank one. 

We present a detailed case study of the exceptional case of ternary quartics, when (d, 2n) = 
(3,4). The Veronese orbitope V^,i = conv(z/4(M 3 )) is a 14- dimensional convex body. Let V^a 
be the 15- dimensional cone over the Veronese orbitope 1^4. As all non-negative ternary 
quartics are sums of squares, we have the following identities of cones: 



3.1 



C 3 4 and 



3,4- 



We next present Reznick's spectrahedral representation of 1^3,4. For this, we identify Sym 4 lR 3 
with its dual, and we introduce coordinates A = (A a ) where the indices a are the exponents 
of monomials in variables x, y, z of degree 4. The ternary quartic corresponding to A is 



(6.1) 



a x ai y a2 z ai 



where (^) = ai!( ^ !Q , ; is the multinomial coefficient. The inner product (qxjq^) = l^ Q (^)^a/ i a 
is S'0(3)-invariant. Given a ternary quartic q x in the notation (16. ip . we associate to it the 
following symmetric 6 x 6-matrix with Hankel structure as in J32J eqn. (5.25)]: 



(6.2) 



( -^400 
A22O 
A202 
A3IO 
A3OI 



A22O 

A040 
A022 

A 130 

A121 



\A211 A 



031 



A 202 
Ao22 

A004 

All2 
Al03 
■^013 



-^310 
Al30 
All2 
A22O 
A2II 
Al21 



^301 

Am 

-^103 
A2II 
A202 
All2 



A21A 

A031 

-^013 
Al21 
All2 
Aq22/ 



Theorem 6.1. The Veronese orbitope V34 is a spectrahedron. It consists of all positive 
semidefinite Hankel matrices K\ as in (16. 2p that satisfy the equation 



(6.3) 



A40O + ^040 + -^004 + 2A220 + 2A202 + 2Aq22 — 1- 
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Proof. It is shown in [32| Ch. 5] that the quartic q\ is non-negative if and only if the Hankel 
matrix K\ is positive semidefmite. Furthermore, the equation (16. 3 j) is the affine equation 
(x 2 + y 2 + z 2 ) 2 = 1 which defines the hyperplane containing the orbit ^(S 2 ). □ 

By contrast, the Veronese coorbitope is not a spectrahedron. 

Theorem 6.2. The convex cone of non-negative ternary quartics = V34 is not a spec- 
trahedron. Its facets have dimension twelve and the intersection of any two facets is an 
exposed face of dimension nine. It also has maximal non-exposed faces of dimension nine. 

Proof. We thank Greg Blekherman who explained this to us. The cone £3,4 is full-dimensional 
in the 15-dimensional space Sym 2n (IR d )*. Its facets come from its defining linear inequalities 

/C 3i4 = {q G Sym 2 „(IR d )* | q(p) > Vp G MP 2 } . 

For p G MP 2 , let F p be the facet exposed by the inequality q{p) > 0, which consists of 
those non-negative ternary quartics q that vanish at p. Since the boundary of /C 3j4 is 14- 
dimensional and we have a two-dimensional family of isomorphic facets, we see that each 
facet F p is 12-dimensional. A non-negative form that vanishes at p G MP 2 must also have its 
two partial derivatives (in local coordinates at p) vanish, which gives three linear conditions 
on the facet F p . Concretely, if we take p = [0 : : 1] with local coordinates x, y, then the 
constant and linear terms of the inhomogeneous quartic q(x, y) must vanish. Consequently, 

(6.4) q(x,y) = H(x,y) + C(x,y) + Q(x,y) , 

where H, C, and Q are, respectively, the terms of degrees 2, 3, and 4 in q. These are binary 
forms. Their 3 + 4 + 5 = 12 coefficients parametrize the linear span of the facet F p , showing 
again that F p is 12-dimensional. The quadratic form H(x,y) is the Hessian of q(x,y) at 
p — [0 : : 1] in these coordinates. 

A form q lies in the relative interior of the facet F p if and only if, given any q' which 
vanishes at p along with its partial derivatives, so that it lies in the linear span of F p , there 
is an e > such that q + eg' also lies in F p . These conditions are equivalent to 

(1) q has no other zeros in MP 2 , and 

(2) the Hessian of q at p is a positive definite quadratic form. 

A form q G F p lies in the boundary of F p when one of these conditions fails, that is, either 

(1) q vanishes at a second point p' G MP 2 \ {p}, or 

(2) the Hessian of q has a double root at some point r G MF 2 . 

Faces of type (1) have the form F p R F p ' . These are nine-dimensional and occur in a four- 
dimensional family parametrized by pairs of distinct points in MP 2 . The union of all such 
faces is a semialgebraic subset of dimension 9 + 4 = 13 in the boundary of £3,4. 

Faces of type (2) also have dimension nine. The condition that the Hessian has a double 
root at a point r G MP 1 gives two linear conditions on the coefficients of q. There is an 
additional condition that the cubic part C of q in (16.41) also vanishes at r, for otherwise q 
takes negative values along the line through p corresponding to r. A face of type (2) is the 
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limit of faces F p nF p of type (1) as p' approaches p along the line corresponding to r. These 
faces form a three-dimensional family on which 5*0(3) acts faithfully and transitively. 

Let us now examine the exposed faces of £3,4. Let £ G "^3,4 be a symmetric tensor in the 
dual cone to /C 3i4 . Then £ is the sum of decomposable symmetric tensors, 

£ = Vi{pi) H V va{Ps) , 

and so it supports the face F P1 D • • • fl F Ps . Thus the exposed faces of /C 3i4 are intersections of 
facets, and they consist of non-negative ternary quartics that vanish at a given set of points. 
Since the faces of type (2) are not of this form, they are not exposed. □ 

Our next agenda item is a discussion of the algebraic boundaries of the convex bodies 
and cones discussed above. The algebraic boundary of the cone V 3 ^ is characterized by the 
vanishing of the determinant of the Hankel matrix K\ in (16. 2p . From this we conclude: 

Corollary 6.3. The algebraic boundary of the Veronese orbitope V 3 ,4 is the variety of di- 
mension 13 and degree six which is defined by the linear equation (16. 3p and the Hankel 
determinant det(K\) = 0. The extreme points ofV 3) 4 are precisely the Hankel matrices K\ 
of rank 1 . 

We observed in the proof of Theorem 16.21 that the boundary of K> 3 ^ consists of non- 
negative quartics q that vanish at some point p of MP 2 , and that the partial derivatives of 
q necessarily also vanish at p. That is, the plane quartic curve defined by q = is singular 
at p. Thus the algebraic boundary of /C3 4 consists of singular ternary quartics. Working in 
P(Sym 4 C 3 ) ~ P 14 , and its dual space of ternary quartics, this algebraic boundary is seen to 
be the dual variety to the Veronese surface which consists of rank 1 Hankel matrices K\. 

Corollary 6.4. The algebraic boundary of the coorbitope cone K, 3 ^ is an irreducible hyper- 
surface of degree 27. Its defining polynomial is the discriminant A q of the ternary quartic 

q(x, y, z) = c mo x 4 + c 3W x 3 y + c 301 x 3 z + c 220 x 2 y 2 + c 211 x 2 yz + c 202 x 2 z 2 + c 130 xy 3 
+ c 121 xy 2 z + c U2 xyz 2 + c W3 xz 3 + c 04 o2/ 4 + c 031 y 3 z + c 022 y 2 z 2 + c 013 yz 3 + c 00 4^ 4 • 

The discriminant A q is a homogeneous polynomial of degree 27 in the 15 indeterminates 
Cijk- In what follows we shall present an explicit expression for A q . That expression will be 
derived from a beautiful classical formula due to Sylvester which can be found in Section 
3.4.D, starting on page 118, of the book by Gel'fand, Kapranov, and Zelevinsky |14j . 

According to [TH Prop. 1.7, page 434], the discriminant A q is proportional to the resultant 
R 3 (q x , q y , q x ) of the three partial derivatives of the quartic q. Here R 3 denotes the resultant 
of three ternary cubics, and the precise relation is A q = 4~ 7 • R 3 (q x , q y , q x ). 

We write (M 3 )* for the space of linear forms on M 3 , and we introduce the linear map 

(6.5) T : (R 3 )* © (R 3 )* © (R 3 )* — ► Sym 4 (M 3 )* , (f,g,h)^fq x + gq v + hq z . 

Next, for an exponent vector a = (a 2 , a 2 , a 3 ) of degree a\ + a 2 + a 3 = 2 and any variable 
t G {x,y,z}, we choose a decomposition of the cubic partial derivative 

(6.6) q t = x ai+1 pW + y a2+1 Q^ + z a3+1 R^ , 



30 



RAMAN SANYAL, FRANK SOTTILE, AND BERND STURMFELS 



where Pa, Qa , and pSa are forms of degree 2 — a±, 2 — a 2 , and 2 — 0:3, respectively. Then 



D n = det 



-La \c£a -Tta 

p(y) p(») 

-fa -fl-a 

pW r( 2 ) , 

\ -i a \c£a -fta / 



is a quartic polynomial. Finally, we define a linear map .D : Sym 2 K 3 — y Sym 4 (IR 3 )* by sending 
5 a i — y D a , where {5 a } is the basis dual to the monomial basis of Sym 2 (M 3 )*. 

Proposition 6.5. (Sylvester [HJ §3.4.D]) The discriminant A q is proportional to the resul- 
tant of the ternary cubics q x ,q y ,q z which is equal to the determinant of the linear map 

T®D: (M 3 )*©(R 3 )*©(R 3 )*©Sym 2 (M 3 )* — ► Sym 4 (R 3 )*. 

This is an irreducible homogeneous polynomial of degree 27 in the 15 coefficients & . 

We write this map explicitly as a 15 x 15 matrix T>(q) whose rows are indexed by the 15 
monomials x l y^z k of degree % + j + k = 4 and whose columns are indexed by 15 auxiliary 
quartics, and whose entry in a given row and column is the coefficient of that monomial in 
that auxiliary quartic. Nine of the quartics come from the map T in ( 16. 5p . They are 

xq x , yq x , zq x , xq y , yq y , zq y , xq z , yq z , zq z . 

The other six are the polynomials -D 002 , D020, -D 2 oo ; -Duo, -Dioij and D ou from D. We only 
describe -D002 and -Duo as the others may be recovered from these by symmetry. 

For Dqq2, note that each partial derivative of q has six terms divisible by x, three divisible 
by y and not x, and a unique term involving z 3 . This leads to a decomposition ( 16. 6p . and 
-D002 is the determinant of the 3x3 matrix: 

4c 4 oo^ 2 + 3c 310 xy + 3c 301 xz + 2c 220 y 2 + 2c 2 n2/2 + 2c 202 z 2 c 130 y 2 + c 1%x yz + c 112 z 2 c 103 
c 310 x 2 + 2c 220 xy + C211XZ + 3c 130 y 2 + 2c 12 iyz + c U2 z 2 4c 04 o2/ 2 + 3c 03 iyz + 2c 022 z 2 c i 3 
cm\x 2 + c 2U xy + 2c 202 xz + c X2 \y 2 + 2c U2 yz + 3c W3 z 2 c 031 y 2 + 2c 022 yz + 3c 013 z 2 4c 00 4 

By a similar reasoning, we find that -Duo is the determinant of the 3x3 matrix: 

4c 400 a; + 3c 310 y + 3c 301 z 2c 220 x + c 130 y + c 121 z 2c 2U xy + 2c 202 xz + c U2 yz + c W3 z 2 
c 3W x + 2c 220 y + c 2U z 3c 130 x + 4c 040 2/ + 3c 03 i2; 2c 121 xy + c U2 xz + 2c 022 yz + c m3 z 2 
_ C301X + c 2U y + 2c 202 z c 121 x + c 031 y + 2c 022 z 2c U2 xy + 3c W3 xz + 3c 013 yz + 4:C 004 z 2 ^ 

This concludes our discussion of the algebraic boundary of the coorbitope cone /C 3i4 . 

We close with the remark that the notations /C.,. and C, t , are consistent with those used 
in the paper [35] where these cones consist of concentration matrices and sufficient statistics 
of a certain Gaussian model. 
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7. Grassmann Orbitopes 

The Grassmann orbitope Q d<n is the convex hull of the Grassmann variety of oriented d- 
dimensional linear subspaces of M. n in its Pliicker embedding in the unit sphere in A^]R n . 
Equivalently, this is the highest weight orbitope for the group SO(ri) acting on A^R": 

Q dtn = conv(SO(n) ■ e 12 ... d ) where e 12 ... d = e 1 Ae 2 A---Ae d G A d R n . 

Faces of the Grassmann orbitope are of considerable interest in differential geometry since, 
according to the Fundamental Theorem of Calibrations, they correspond to area-minimizing 
(^-dimensional submanifolds of M. n . References to this subject include the seminal article on 
calibrated geometries by Harvey and Lawson [16J and the beautiful expositions by Morgan 
[TT1 126] . In this section we review basic known facts about Q djU and we initiate its study 
from the perspectives of combinatorics, semidefmite programming, and algebraic geometry. 
Vectors in A^IR™ are written in terms of Pliicker coordinates relative to the standard basis: 

P = ^ ^ Phiz— id e n A e *2 A • • • A &i d . 
l<ii<"'<irf<n 

The Pliicker vector p lies in the oriented Grassmann variety if and only if it is decomposable, 
i.e. p = ui A u 2 A ■ ■ ■ A u d for some pairwise orthogonal subset {m, u 2 , ■ ■ ■ , u d } of S n_1 . This 
happens if and only if p lies in the unit sphere in A d M. n and satisfies all quadratic Pliicker 
relations, the relations among the rfxrf-minors of a dxn- matrix. These relations generate 
the prime ideal called the Pliicker ideal I d ^ n . Thus the oriented Grassmann variety is the 
algebraic subvariety of A^IR n defined by the ideal 

(7-1) /*»+(!- E 

l<ii <---<i [ j<n 

The convex hull of that real algebraic variety is the Q) -dimensional Grassmann orbitope Ga,n- 

Example 7.1 (d—2, n=4). The Grassmann orbitope Q 2 ,a is the convex hull of the variety of 

(7.2) < p 12 p 3A - p 13 p 24 + p u p 23 , p\ 2 + p 2 13 + p 2 u + pl 3 +pl 4 +Pu-l). 

As suggested by [261 Proposition 3.2], we perform the orthogonal change of coordinates 

u = -^(Pi2 +P34), v = ^(pi 3 -P24), ^ = 73(^14+^23), 

x = 73(^12 -P34), V = 73(^13 +P24), z= 7=(pu -Pas)- 

This is simultaneous rotation by tt/4 in each of the coordinate planes spanned by the pairs 
(Pi2)P34)> (P135P24), and (j?i4,P23)- I n these new coordinates, the prime ideal (17.21) equals 

/ 2 1 2 1 2 1 2,2,2 1 \ 
\u +v +w - - , x +y + z - - ). 

This reveals that Q 2 ^ is the direct product of two three-dimensional balls of radius l/y/2. □ 

We next examine the case d = 2 and arbitrary n. The vectors p in A 2 M. n can be identified 
with skew-symmetric nxn- matrices, and this brings us back to the orbitopes in Section 3.2. 
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Corollary 7.2. The Grassmann orbitope Q^ n coincides with the skew-symmetric Schur-Horn 
orbitope of a skew- symmetric matrix N £ having rank two and A(N) = (1, 0, 0, . . . , 0). 

If p is a real skew-symmetric matrix whose eigenvalues are ±zAi, . . . , ±iAfc, where i = %/— T, 
then the matrix % • p is Hermitian and its eigenvalues are the real numbers ±Ai, . . . , ±A&. 
Recall that the operator Ck computes the k-th additive compound matrix of a given matrix. 

Theorem 7.3. Let n > 5 and k = [n/2\. The Grassmann orbitope equals the spectrahedron 

(7.3) Q 2 , n = {peA 2 R n : Id^-A(i-p) ^0}. 

ite algebraic boundary d a Qi, n is an irreducible hypersurface of degree 2 k , defined by a factor of 
the determinant of the matrix Id/ n \ — Ck{i-p). The proper faces of @2,n are SU(m)-orbitopes 

for 1 < m < k. Every face F is associated with an even- dimensional subspace Vf equipped 
with an orthogonal complex structure and the extreme points of F correspond to complex 
lines in V F . 

Everything in this theorem is also true for the small cases n = 3,4, with the exception 
that the quartic hypersurface <9 a (?2,4 is not irreducible, as was seen in Example 17.11 In light 
of Theorem I3.11[ the reducibility of £2,4 arises because the two-dimensional crosspolytope 
equals the square, which decomposes as a Minkowski sum of two line segments. This may 
also be seen as the stabilizer of a decomposable tensor in 50(4) is ±7, where I is the identity 
matrix, and SO{A)/{±I} ~ SO{3) x SO(3), so £ 2 ,4 is also an orbitope for SO(3) x SO(3). 

Proof. We begin with the last statement about the face lattice of Q2, n - This result is well- 
known in the theory of calibrations, where it is usually phrased as follows: every face of the 
Grassmannian of two-planes in MJ 1 consists of the complex lines in some 2m- dimensional 
subspace of W 1 under some orthogonal complex structure. See §1.1] . 

To derive the spectrahedral representation (|7.3p . we note that the eigenvalues of C^ii ■ p) 
are the sums of any k distinct numbers of the eigenvalues of the skew-symmetric matrix p. 
These are — Ai, . . . — A&, Ai, . . . , A*, if n is even and — Ai, . . . , — 0, Ai, . . . , if n is odd. In 
light of Corollary 17. 2\ we can apply the results in Section 3.2 to conclude that p lies in @2,n if 
and only if ±Ai ± A2 ± • ■ ■ ± A& < 1 for all choices of signs. In terms of polyhedral geometry, 
this condition means that the vector A(p) = (Ai, A2, . . . , Ajt) lies in the crosspolytope A((? 2 ,n)- 
Since all m eigenvalues of Ck(i ■ p) are bounded above by the maximum of the 2 k special 
eigenvalues ±Ai ± A 2 ± ■ ■ ■ ± A&, we conclude that p £ Q 2 ,n if and only if Id^ — Ck{i-p) b 0. 

To compute the algebraic boundary d a Q2,n we consider the expression 

! J (1 +<tx\i + 0~ 2 A 2 H ha fc A fc ). 

cre{-l, + l} fc 

This is a symmetric polynomial of degree 2 k ~ l in the squared eigenvalues A^, . . . , A^, and 
hence it can be written as a polynomial in the coefficients of the characteristic polynomial 

det(i -p- x ■ Idn) = x nmod2 • (x 2 - \\){x 2 - X 2 2 ) ■ ■ ■ (x 2 - \ 2 k ). 



ORBITOPES 



33 



The resulting polynomial has degree 2 k in the entries p a b of the matrix p, and it vanishes on 
the boundary of the orbitope Qi,n- It can be checked that it is irreducible for n > 5. □ 

Example 7.4. Let n = 6 and consider the characteristic polynomial of our Hermitian matrix: 



det 



/ 


—x 


ip\2 


IP13 


ipu 


ipi5 


ipie\ 






—x 


iP23 


IP2A 


W25 


W26 




-iPi3 


-IP23 


—x 


iP34 


IP35 


IP36 




-ip u 


-IP24 


-IP34 


—x 


ipis 


ipm 




"«Pl5 




-W35 


iP45 


—x 


iP56 


V 


-ipw 


-IP26 


~iP36 


IP46 


-IP56 


—x) 



x % + a 4 x 4 + a 2 x 2 + a 



(* 2 -A^)(* 2 -A 2 )(* 2 -A 2 ) 



The algebraic boundary of the Grassmann orbitope Q 2 ,& is derived from the polynomial 

Y\_ (1 + 0i Ai + (T2A2 + cr 3 \ 3 ) = 04+404— 8a 2 a2+16a 2 — 16a 4 a2+6a 2 +64ao— 8a 2 +4a4+l . 
<re{±i} 3 

We rewrite this expression in terms of the 15 unknowns pij to get an irreducible polynomial 
of degree 8 with 10791 terms. This is the defining polynomial of the hypersurface d a Q2,e- For 
n = 7 we use the same polynomial in ao,a 2 , 04 but now there is one more eigenvalue 0. The 
defining polynomial of <9 a {?2,7 has 44150 terms of degree 8 in the 21 matrix entries p^. □ 

We now come to the harder case d = 3, n = 6. The Grassmann orbitope £/ 3)6 is a 20- 
dimensional convex body Its facial structure was determined by Dadok and Harvey in [9], 
and independently by Morgan in [26]. We present their well-known results in our language. 

Theorem 7.5. The orbitope Q 3 fi has three classes of positive-dimensional exposed faces: 

(1) one- dimensional faces from pairs of subspaces that satisfy the angle condition ( |7.^[ ); 

(2) three- dimensional faces that arise as SO(3)-orbitopes and these are round 3-balls; 

(3) 12- dimensional faces that are SU(3)-orbitopes, from the Lagrangian Grassmannian. 

Harvey and Morgan [17] extended these results to Q 3 j. We will here focus on d = 3, n = 6. 
Our first goal is to explain the faces described in Theorem 17. 5[ starting with the edges. Let L 
and L' be three-dimensional linear subspaces in M 6 with corresponding unit length Plucker 
vectors p and p' . We define 6i(L,L') to be the minimum angle between any unit vector 
V\ £ L and any unit vector W\ G V . Next, 02{L, V) is the minimum angle between two unit 
vectors v 2 G L and w 2 G L' such that v 2 _L v\ and w 2 J- w\. Finally, we define 9 3 (L,L') 
to be the angle between two unit vectors v 3 G L and w 3 G L' such that v 3 _L {^1,^2} and 
w 3 _L {wi, w 2 }. We refer to [26], Lemma 2.3] for the fact that 62 and 6 3 are well defined by 
this rule. The angle condition referred to in part (1) of Theorem 17.51 is the inequality 

(7.4) 9 3 (L,L') < 6 1 (L,L') + 6 2 (L,L'). 

The convex hull of p and p' is an exposed edge of Q 3 $ if and only if the condition (I7.4p holds. 

The maximal facets in part (3) of Theorem 17.51 are known to geometers as special La- 
grangian facets, and we represent them as orbitopes as follows. The special unitary group 
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SU(3) consists of all complex 3 x 3-matrices U with det(U) = 1 and UU* = Id%. If A = re(U) 
and B = im(U), so that U = A + iB, then we can identify U with the real 6x6-matrix 



Note that this matrix lies in 50(6) if and only if AA T + BB T = Id 3 , AB T = BA T , and 
det(A + iB) = 1. Hence the transformation U y-¥ U realizes SU(3) as a subgroup of S0(6). 

The orbit SU(3) ■ eiAe2Ae3 is known as the special Langrangian Grassmannian. This is 
a five-dimensional real algebraic variety in the unit sphere in A3R 6 . We call its convex hull 



the special Lagrangian orbitope. It is obtained from the 20-dimensional Grassmann orbitope 
(?3 i6 by maximizing the linear function P123 — P156 + P246 — P345, which takes value 1 on that 
face. The face SCs t e is 12-dimensional because it satisfies the two affine equations 

P123 - Pi56 + P2W ~ P345 = re(A + iB) = 1 and p 126 - P135 + P234 ~ P456 = im(A + iB) = 
plus the six independent linear equations that cut out the Langrangian Grassmannian: 

P125 + Pl36 = P134 + P235 = P124 - P236 = Pl4G + P25G = P245 + £>346 = Pl45 ~ P356 = 0. 

The exposed faces of type (2) in Theorem 17.51 are not facets. Each of them is the inter- 
section of two special Lagrangian facets. For example, consider the two linear functionals 



which were discussed in [26| §4.4]. Each of them supports a special Lagrangian facet. The 
intersection of these two 14- dimensional facets is a three-dimensional ball. Indeed, the linear 
functional \{4>+ + 0-) = P123 — P156 is bounded above by 1 on Q 3> q, and the subset at which 
the value equals 1 is contained in the linear span of the six basis vectors e\ A A ej where 
i,j G {2,3,5,6}. In the intersection of this linear space with Q^q we find the Grassmann 
orbitope £2,4 from Example I7.1[ with our linear functional P123 — P156 being represented by 
the scaled coordinate y/2x. The face of £2,4 where \/2x attains its maximal value 1 is a 
3-ball. This concludes our discussion of the census of faces given in Theorem 17.51 

A natural question that arises next is whether the Grassmann orbitope C/3 g or its dual 
body Q^ q can be represented as a spectrahedron. It turns out that the answer is negative. 

Theorem 7.6. The Grassmann coorbitope has extreme points that are not exposed. The 
Grassmann orbitope Q^fi has edges that are not exposed. Neither of them is a spectrahedron. 

Proof. The first assertion was proved by Dadok and Harvey in [9j Theorem 7]. For the second 
assertion we proceed as follows. We apply the technique in [261 Lemma 2.2] and restrict to 
the linear subspace V = M{ei 2 3, ei26, ei 35 , e 2 34, ei 56 , e 246 , e 345 , e 456 } of A3M 6 . The intersection 
°f £3,6 H V is the SO (2) x SO (2) x S'0(2)-orbitope of ei 2 3, where the action is by unitary 
diagonal 3x3-matrices, while the intersection iS£ 3i6 fl V is the 5*0(2) x S'0(2)-orbitope of 
ei.23, where the action is by special unitary diagonal 3x3-matrices, as described in [261 §4-3]. 




iS>£ 3)6 = conv(Si7(3) ■ eiAe 2 Ae 3 ) 



'+ 



P123 - Pi56 + £>246 - £>345 and 



P123 -P156 -£>246 +P345, 
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We claim that the SO (2) x S'0(2)-orbitope ££3,6 H V is 2-neighborly. This can be seen 
by examining the bivariate trigonometric polynomials of the form 

/ = £123 • cos(a)cos(/9)cos(— a — j3) + xyi% ■ cos(a)cos(/3)sin(— a — j3) 
— X135 ■ cos(a)sin(/3)cos(— a — (3) + X234 • sin(a)cos(/3)sin(— a — j3) 
+ Xi5 6 • cos(a)sin(/3)sin(— a — f3) — x 2 46 ■ sin(a)cos(/3)sin(— a — (3) 
+ £345 • sin(a)sin(/9)cos(— a — (3) — x 456 • sin(a)sin(/3)sin(— a — f3). 

Indeed, for any choice of (ai,/?i) and (a 2 ,/3 2 ), we can find eight real coefficients x^k such 
that / and its derivatives vanish at (ai,/3i) and (0:2, As) but / is strictly positive elsewhere. 

The results in imply that every exposed edge of ^ 3j6 fl V is also an exposed edge of 
Q 3 fi. We believe that Morgan's technique can be adapted to show that every exposed edge 
of SCzfi PI V is an exposed edge of SC^^. For the purpose of proving Theorem 17. 6[ however, 
we only need to identify one exposed edge of ££3,6 H V that is an exposed edge of ££3,6 • 

The claim that such exposed edges exist can be derived from [91 Theorem 12 (i)]. With 
the help of Philipp Rostalski, we also obtained a computational proof of that claim. This 
was done as follows. We selected various random choices of points (oti,/3i) and (0(2,^2) in 
IR 2 . Each choice specifies two three-dimensional subspaces L and V of R 6 for which equality 
holds in the angle condition f)7.4p . The corresponding line segment is not an exposed edge 

of £ 3 ,6- 

To show that the line segment between L and L' is exposed in <S£ 3i6 , we use a technique 
from semidefinite programming. First we compute the eight coefficients Xijk of the supporting 
function / as above. This gives a linear function Yl x ijkPijk on A3M 6 . We then run a first- 
order Lasserre relaxation (cf. [21]) to minimize this linear function subject to the linear and 
quadratic constraints that cut out the special Lagrangian Grassmannian. The optimal value 
is zero, the optimal Lasserre moment matrix has rank two, and its image in A3M 6 lies in 
the relative interior of the line segment between L and V . We then re-optimize for various 
perturbations of the linear function ^ijkVijk- The output of each run is a rank one moment 
matrix which certifies either L or V as optimal solution of the optimization problem. This 
proves that the face of ££3^ exposed by XijkPijk is the line segment between L and V . □ 

Theorem 17.61 shows that Grassmann orbitopes are generally not spectrahedra. We do not 
know whether they are linear projections of spectrahedra. 
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